Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge

Chase Fensore; Eugene Agichtein; Joyce C Ho; Kaustubh Dhole

REVIEW 4 major objections 5 minor 14 references

Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reranking with RankLLaMA lifts retrieval MAP from 0.523 to 0.797 but costs 84 seconds per question, while vocabulary alignment between question and document is the strongest predictor of RAG answer quality.

desk verdict A candid, useful competition write-up whose headline analytic claim about vocabulary alignment outruns its statistics. read the letter →

arxiv 2506.22644 v1 pith:EISUTJWK submitted 2025-06-27 cs.CL cs.IR

Chase Fensore , Kaustubh Dhole , Joyce C Ho , Eugene Agichtein This is my paper

classification cs.CLcs.IR

keywords retrieval-augmentedgenerationhybridretrievalneuralre-rankingLiveRAGChallengeDataMorganavocabularyalignmentfaithfulnesspromptoptimization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This systems paper reports what did and did not work in building a retrieval-augmented generation pipeline for the LiveRAG Challenge 2025, where systems answer questions over the FineWeb-10BT corpus. The authors' central claim is that neural re-ranking with RankLLaMA is the most effective retrieval upgrade, lifting mean average precision from 0.523 to 0.797 on their 200-question development set, but at roughly 84 seconds per question, which made it infeasible for the challenge's two-hour, 500-question window. They also find that vocabulary alignment between question and document is the strongest predictor of answer quality, and that DSPy-optimized prompts raise semantic similarity while driving refusal rates to zero, which they interpret as over-confidence rather than robustness. The submitted hybrid system without reranking placed 4th in faithfulness and 11th in correctness among 25 teams, a gap the authors attribute to a conservative prompting strategy that refuses too often.

What carries the argument

The carrying mechanism is a hybrid retrieval pipeline: BM25 sparse scores and E5 dense cosine scores are normalized and fused to select the top ten chunks, which are passed to Falcon3-10B-Instruct for answer generation. On top of this, the paper optionally places RankLLaMA-7B, a pointwise generative reranker that scores each retrieved passage independently before selection; this component produces the large retrieval and answer-quality gains. The DataMorgana development set supplies the experimental structure, with 64 question-user combinations spanning factuality, premise, phrasing, linguistic variation, and user expertise, and it is this category structure that allows the authors to attribute performance differences to vocabulary alignment and user level.

What would settle it

Label the 500-question LiveRAG test set with DataMorgana's categories and run the same hybrid system without reranking: if document-distant questions do not show a substantially larger cosine-similarity and refusal-rate deficit than document-similar questions, the vocabulary-alignment claim fails to transfer.

Watch

Extended reading notes

Core claim

On the DataMorgana-generated development set, the paper demonstrates that a hybrid of BM25 sparse and E5 dense retrieval, fused by normalized score combination, matches sparse retrieval alone on retrieval metrics but provides a better base for downstream generation. Adding RankLLaMA as a pointwise reranker improves MAP from 0.523 to 0.797, a 52% relative gain, raises ROUGE-1 from 0.368 to 0.403, and cuts the refusal rate from 17% to 3.5%, but increases per-question time from 1.74 to 84 seconds. The category analysis shows the largest quality gap is vocabulary alignment: questions phrased like the source documents reach 0.762 cosine similarity and 9.4% refusals, while document-distant phrasing falls to 0.562 and 25.5% refusals. The authors argue from these results that retrieval quality dominates RAG answer quality on this corpus, that generative reranking is effective but computationally prohibitive, and that dev-set prompt optimization can overfit semantic-similarity metrics while destroying refusal calibration.

Load-bearing premise

The 200 synthetic questions in the development set, with their fixed 80/20 expert-to-novice split and specific category mix, are representative enough of the unseen LiveRAG test questions and of real-world RAG queries that the measured performance gaps and category rankings transfer.

Editorial extensions

If this is right

Where compute is not constrained, adding RankLLaMA-style reranking to hybrid retrieval should improve answer faithfulness, because the reranked pipeline raised MAP from 0.523 to 0.797 and cut refusals from 17% to 3.5%.
Because vocabulary alignment shows the largest performance gap (cosine similarity 0.762 vs 0.562), query rewriting or document expansion that closes lexical distance should improve RAG quality more than further retrieval-architecture changes.
Prompt optimization that maximizes only semantic similarity can eliminate appropriate refusals, so production RAG systems should track refusal rate and out-of-distribution calibration alongside dev-set similarity metrics.
The faithfulness-versus-correctness ranking gap (4th vs 11th) implies that conservative prompting grounded answers but left many answerable questions unanswered, making adaptive refusal thresholds a direct next step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The 80/20 expert-to-novice split in the synthetic development set is a modeling choice; a more balanced split might shrink the reported expert advantage (0.709 vs 0.628 cosine similarity) and could change which system variant wins a leaderboard.
The 84-second reranking cost is hardware- and batching-dependent; with batched inference or a smaller reranker, the same MAP gain could fit inside the challenge's two-hour window, so the 'prohibitive' conclusion is contingent on the specific deployment.
The vocabulary-alignment result suggests a testable extension the paper does not run: train a query-to-document paraphrase model on the DataMorgana category labels and check whether document-distant questions approach the 0.762 cosine similarity of document-similar ones.
If future challenge scoring weights refusals more heavily than wrong answers, the conservative hybrid submission would become less attractive and the DSPy-optimized zero-refusal prompts could become the stronger choice despite their calibration concerns.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick read on the LiveRAG write-up. It is a clean competition report with a few genuinely usable numbers and one analytical claim that goes beyond what the data support.

What is actually useful: the RankLLaMA reranking numbers (MAP 0.523 to 0.797, 84s vs 1.74s per question) give a concrete cost/benefit data point for pointwise reranking in a time-bounded evaluation. The doc2query failure and the DSPy over-confidence results are about what you would expect, and the team's decision to stick with the conservative prompt for the actual submission shows sensible judgment. The paper is also appropriately honest about its own limitations, including the small n=200 dev set.

Soft spots, in order. First, the "vocabulary alignment is the strongest predictor" claim is not supported by the analysis as presented. It rests on one-dimensional marginals from a DataMorgana design where the category dimensions are likely correlated (64 combinations drawn into 200 items, with an 80/20 expertise split). No regression, no interaction analysis, no significance test, and the best-versus-worst comparison uses n=5 and n=6. At most the data show a strong association in the marginals, not that it is the strongest predictor. Second, the DSPy evaluation appears to be in-sample: prompts were optimized on a 160/40 split, but the reported numbers in Table 2 are for the full 200-question set, which means the comparison is not held out. The paper should say this explicitly or re-run the evaluation on the held-out 40. Third, there are no confidence intervals or significance tests anywhere, so the system rankings are credible but not statistically grounded. No code or data are released, which limits replication.

None of this sinks the paper for what it is: a competition report. The core numbers are internally consistent and the authors are honest about scope. For a workshop or short-paper track, this is fine. The abstract and conclusion should soften "strongest predictor" to something like "most strongly associated in our marginal analysis," and the in-sample DSPy issue should be disclosed in the methods. Who gets value from this: practitioners wanting sanity-check numbers on reranking cost and prompt-optimization risk, and anyone documenting a live RAG challenge submission. It does not need a full journal referee, but it deserves a serious short-paper review, conditional on the claim language being fixed.

Referee Report

4 major / 5 minor

Summary. The paper describes a hybrid retrieval-augmented generation system submitted to the LiveRAG Challenge 2025, combining BM25 sparse retrieval, E5 dense retrieval, score fusion, optional RankLLaMA pointwise re-ranking, Falcon3-10B-Instruct answer generation, and DSPy prompt optimization. The authors generated 200 DataMorgana questions spanning 64 question/user category combinations and report that re-ranking improves MAP from 0.523 to 0.797 but costs about 84 seconds per question, making it infeasible under the challenge's two-hour constraint. They also report that DSPy-optimized few-shot and chain-of-thought prompts achieve higher semantic similarity (0.771 and 0.756) than the manually engineered prompt (0.668) but exhibit 0% refusal rates, which they interpret as over-confidence. The submitted hybrid system placed 4th in faithfulness and 11th in correctness among 25 teams. A stratified analysis in Section 3.3 is used to claim that vocabulary alignment between questions and documents is the strongest predictor of performance, with document-similar phrasing improving cosine similarity from 0.562 to 0.762.

Significance. If the empirical claims hold, the paper provides useful competition-grounded evidence about the cost-effectiveness of generative re-ranking for RAG and about the sensitivity of answer quality to query-document vocabulary overlap. The manuscript is transparent about challenge rankings, measured latency, refusal behavior, and the fidelity/correctness trade-off, which are strengths. The paper does not ship code or checkable artifacts, so reproducibility rests on the textual description. The main limitation is that the headline analytical claims are not backed by formal statistical inference or controlled comparisons, and at least one data-composition statement conflicts with the reported per-category counts. These issues are addressable within the scope of the manuscript but currently limit confidence in the conclusions.

major comments (4)

[§3.3, Table 3] The claim in the abstract and conclusion that vocabulary alignment was the strongest predictor of performance is not supported by the reported analysis. The supporting numbers (cosine similarity 0.762 vs 0.562, ROUGE-1 0.431 vs 0.296) are univariate marginals over the Linguistic Variation category, with no control for Question Factuality, Question Premise, Question Phrasing, or User Expertise, even though DataMorgana generates these dimensions jointly and the 200-item sample may contain correlated category assignments. No regression, ANOVA, permutation test, or confidence interval is reported, so the relative importance of vocabulary alignment is an informal reading of one table. The extreme-case comparison in the same section uses n=5 and n=6 and is too small to be evidence. The Limitations section (3.5) notes the small dev set but does not address this confounding; please either replace the claim with a controlled analysis or explicitly soften it to a descriptive observation.
[§3.2, Table 2 and §2.3] The comparison between DSPy-optimized prompting and the base prompt is not shown to be a held-out comparison. The optimization uses a 160/40 (80/20) train/validation split of the 200-question set, and the reported few-shot (0.771) and CoT (0.756) cosine similarities appear to be the scores associated with prompts selected during optimization on that validation material, while the base-prompt 0.668 is measured on the full set. The text states that the optimized methods 'significantly outperform' the baseline, but no separate test fold, no cross-validation summary, and no significance test is described. Because prompt optimizers can overfit small development sets, the generalizability conclusion and the 0%-refusal calibration concern should be evaluated on an untouched held-out set before being presented as findings.
[§2.1 vs. Table 3] The data-composition description contradicts the reported counts. Section 2.1 states that user expertise categories contain 80% expert and 20% novice users, but Table 3 lists Novice (n=101) and Expert (n=99), which is approximately 50/50. Please reconcile this discrepancy or correct one of the statements. The user-expertise analysis in Section 3.3 depends on knowing the actual composition, and the mismatch undermines confidence in the data-generation description.
[§3.1, Table 1] The central empirical claim that re-ranking 'improves MAP from 0.523 to 0.797' is reported as a single point estimate per system, with no confidence intervals, bootstrap estimates, or paired significance tests across the 200 questions. Retrieval metrics such as MAP and nDCG@10 can be sensitive to a small number of difficult queries, and the latency comparison (84.37s vs 1.74s per question) also lacks any variance information. Please add uncertainty quantification, or state explicitly that the improvements are descriptive, so that the abstract's use of 'demonstrate' matches the evidence provided.

minor comments (5)

[§3.3] The phrase 'we observe significant performance variations' uses 'significant' in a statistical sense, but no significance tests are reported; consider using 'substantial' or adding the tests.
[§2.2] The hybrid retrieval description says 'selecting top 10 documents based on normalized score fusion,' but earlier the sparse and dense retrievers each retrieve k=30 documents; please clarify whether the final prompt contains 10 chunks or 30 and where the fusion cutoff is applied.
[§3.2] The table caption for Table 1 says 'Time indicates the mean time required for retrieval and generation, in seconds,' but Table 1 is presented as a retrieval-performance table; clarify whether the time includes answer generation or only retrieval.
[§3.4] The sentence attributing the correctness/faithfulness gap to 'overly cautious' behavior is plausible but speculative; the refusal-rate evidence from the dev set (17%) is indirect, and the test set is not analyzed for refusals.
[§1] The introduction says the dev set 'mirrors real-world QA diversity,' but the 80/20 expert/novice split, if correct, is a specific design choice; consider citing a source or qualifying this claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports empirical measurements on an external benchmark and external tools; the only self-citation is a non-load-bearing software reference.

full rationale

The paper's central claims are empirical evaluations of a RAG pipeline on a synthetic development set generated by DataMorgana and on the LiveRAG Challenge test set. The retrieval, re-ranking, and generation results (MAP, ROUGE, BLEU, cosine similarity, refusal rates) are measured quantities, not consequences of definitions or fitted parameters. The 'vocabulary alignment' finding is an observational comparison of a controlled DataMorgana question category, not a quantity defined in terms of the systems' outputs; its statistical weakness is a validity concern, not circularity. The only self-citation, Reference [1] (PyTerrier-GenRank by author Kaustubh Dhole), is a software implementation tool for pointwise re-ranking and does not supply any load-bearing scientific premise, uniqueness theorem, or ansatz. All other citations are to external tools (DataMorgana, DSPy, Falcon, RankLLaMA, evaluation metrics) and are independent of the paper's own results. The paper is self-contained against an external benchmark, and its conclusions do not reduce to their inputs by construction.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claims rest on assumptions about the representativeness of the synthetic dev set and the validity of automatic metrics, plus several manually chosen system hyperparameters (chunk size, k, fusion cutoff, decoding parameters, dev set size). None of these are derived or externally benchmarked; they are design choices. No new theoretical entities are introduced.

free parameters (6)

Chunk size = 512 tokens
Chosen for sentence-aware splitting; affects retrieval granularity and is not optimized or justified.
Retrieval depth k = 30 per index
Top-30 documents from BM25 and from E5; chosen by hand, affects fusion and reranking input.
Fused top-10 cutoff = 10
After normalized score fusion, top 10 documents are selected; selection threshold is ad hoc.
Generation temperature / top_p = 0.6 / 0.9
Used for Falcon3-10B-Instruct decoding; chosen by manual tuning without ablation.
DSPy train/val split = 160/40
Used for few-shot bootstrap and MIPROv2 optimization; exact split affects optimized prompt performance.
Development set size = n=200
Chosen due to time constraints; small sample limits statistical power for category-level claims.

assumptions (4)

domain assumption DataMorgana-generated synthetic questions reflect the diversity of LiveRAG test questions and real-world RAG queries.
Invoked in Sections 2.1 and 3.3; if false, the category analysis and the 'strongest predictor' conclusion do not transfer to the challenge test set.
domain assumption Automatic metrics (ROUGE, BLEU, MiniLM cosine similarity, refusal rate) are valid proxies for the challenge's correctness and faithfulness judgments.
Section 3.2 uses these metrics to compare prompting strategies and to interpret 0% refusal as overconfidence rather than better answers.
domain assumption The FineWeb-10BT corpus is the appropriate retrieval corpus, and the doc2query model's domain mismatch explains its poor performance.
Section 3.1 attributes doc2query failure to corpus mismatch without measuring the enhancement model's coverage.
domain assumption RankLLaMA's pointwise scores for a query-passage pair are comparable across passages for ranking.
Section 2.2 assumes the pointwise prompt (query + passage) gives scores that can rank passages retrieved by hybrid fusion.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge." pith.science (2026). https://pith.science/paper/EISUTJWK

@misc{pith2026250622644,
  author       = {Pith},
  title        = {Pith review of: Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EISUTJWK}},
  note         = {Machine review of arXiv:2506.22644}
}

read the original abstract

We present our submission to the LiveRAG Challenge 2025, which evaluates retrieval-augmented generation (RAG) systems on dynamic test sets using the FineWeb-10BT corpus. Our final hybrid approach combines sparse (BM25) and dense (E5) retrieval methods and then aims to generate relevant and faithful answers with Falcon3-10B-Instruct. Through systematic evaluation on 200 synthetic questions generated with DataMorgana across 64 unique question-user combinations, we demonstrate that neural re-ranking with RankLLaMA improves MAP from 0.523 to 0.797 (52% relative improvement) but introduces prohibitive computational costs (84s vs 1.74s per question). While DSPy-optimized prompting strategies achieved higher semantic similarity (0.771 vs 0.668), their 0% refusal rates raised concerns about over-confidence and generalizability. Our submitted hybrid system without re-ranking achieved 4th place in faithfulness and 11th place in correctness among 25 teams. Analysis across question categories reveals that vocabulary alignment between questions and documents was the strongest predictor of performance on our development set, with document-similar phrasing improving cosine similarity from 0.562 to 0.762.

Figures

Figures reproduced from arXiv: 2506.22644 by the authors.

**Figure 1.** Pointwise Re-ranking Prompt 2.3 Answer Generation We used the Falcon3-10B-Instruct [14] model for answer generation with temperature=0.6 and top_p=0.9. We performed manual prompt engineering and arrived at a base prompt aimed to emphasize conciseness and faithfulness ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

14 extracted references · 3 canonical work pages

[1]

Kaustubh Dhole. 2024. PyTerrier-GenRank: The PyTerrier Plugin for Reranking with Large Language Models . https://github.com/emory-irlab/pyterrier_genrank

work page 2024
[2]

Simone Filice, Guy Horowitz, David Carmel, Zohar Karnin, Liane Lewin-Eytan, and Yoelle Maarek. 2025. Generating Diverse Q&A Benchmarks for RAG Evalua- tion with DataMorgana. doi:10.48550/arXiv.2501.12789 arXiv:2501.12789 [cs]

arXiv doi:10.48550/arxiv.2501.12789 2025
[3]

Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts

Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2023. DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. doi:10.48550/arXiv.2310.03714 arXiv:2310.03714 [cs]

arXiv doi:10.48550/arxiv.2310.03714 2023
[4]

Joon Ho Lee. 1995. Combining multiple evidence from different properties of weighting schemes. In Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR ’95). Association for Computing Machinery, New York, NY, USA, 180–188. doi:10.1145/215206.215358

work page arXiv 1995
[5]

Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74–81

2004
[6]

Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine- Tuning LLaMA for Multi-Stage Text Retrieval. In Proceedings of the 47th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2024) . ACM, 2421–2425. doi:10.1145/3626772.3657951

arXiv 2024
[7]

Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. Document expansion by query prediction. arXiv preprint arXiv:1904.08375 (2019)

arXiv 2019
[8]

Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab

Krista Opsahl-Ong, Michael J. Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. 2024. Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs. doi:10.48550/arXiv. 2406.11695 arXiv:2406.11695 [cs]

arXiv doi:10.48550/arxiv.2406.11695 2024

Show all 14 references

[9]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics . 311–318

2002
[10]

Guilherme Penedo, Hynek Kydlíček, Anton Lozhkov, Margaret Mitchell, Colin A Raffel, Leandro Von Werra, Thomas Wolf, et al . 2024. The fineweb datasets: Decanting the web for the finest text data at scale.Advances in Neural Information Processing Systems 37 (2024), 30811–30849

2024
[11]

Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023. RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze! doi:10.48550/ arXiv.2312.02724 arXiv:2312.02724 [cs]

2023 doi arXiv
[12]

Dilara Soylu, Christopher Potts, and Omar Khattab. 2024. Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together. doi:10.48550/ arXiv.2407.10930 arXiv:2407.10930 [cs]

2024 doi arXiv
[13]

Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2024. Is ChatGPT Good at Search? Investi- gating Large Language Models as Re-Ranking Agents. doi:10.48550/arXiv.2304. 09542 arXiv:2304.09542 [cs]

2024 doi arXiv
[14]

Falcon-LLM Team. 2024. The Falcon 3 Family of Open Models. https: //huggingface.co/blog/falcon3

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Kaustubh Dhole. 2024. PyTerrier-GenRank: The PyTerrier Plugin for Reranking with Large Language Models . https://github.com/emory-irlab/pyterrier_genrank

work page 2024

[2] [2]

Simone Filice, Guy Horowitz, David Carmel, Zohar Karnin, Liane Lewin-Eytan, and Yoelle Maarek. 2025. Generating Diverse Q&A Benchmarks for RAG Evalua- tion with DataMorgana. doi:10.48550/arXiv.2501.12789 arXiv:2501.12789 [cs]

arXiv doi:10.48550/arxiv.2501.12789 2025

[3] [3]

Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts

Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2023. DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. doi:10.48550/arXiv.2310.03714 arXiv:2310.03714 [cs]

arXiv doi:10.48550/arxiv.2310.03714 2023

[4] [4]

Joon Ho Lee. 1995. Combining multiple evidence from different properties of weighting schemes. In Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR ’95). Association for Computing Machinery, New York, NY, USA, 180–188. doi:10.1145/215206.215358

work page arXiv 1995

[5] [5]

Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74–81

2004

[6] [6]

Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine- Tuning LLaMA for Multi-Stage Text Retrieval. In Proceedings of the 47th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2024) . ACM, 2421–2425. doi:10.1145/3626772.3657951

arXiv 2024

[7] [7]

Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. Document expansion by query prediction. arXiv preprint arXiv:1904.08375 (2019)

arXiv 2019

[8] [8]

Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab

Krista Opsahl-Ong, Michael J. Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. 2024. Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs. doi:10.48550/arXiv. 2406.11695 arXiv:2406.11695 [cs]

arXiv doi:10.48550/arxiv.2406.11695 2024