Pith. sign in

REVIEW 4 major objections 6 minor 51 references

Parallel Key-Value Cache Fusion for Position Invariant RAG

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read KV-Fusion makes decoder-only LLMs order-invariant in RAG by encoding each retrieved passage with identical positional information and prefilling a trainable decoder with the fused key-value caches.

desk verdict The order-invariance result is real but largely architectural; the accuracy gains over baselines are confounded by a missing same-data fine-tuned control. read the letter →

arxiv 2501.07523 v2 pith:KDVCOVHJ submitted 2025-01-13 cs.AI cs.CL

Philhoon Oh , Jinwoo Shin , James Thorne This is my paper · ORCID
classification cs.AIcs.CL
keywords key-valuecachepositioninvarianceretrieval-augmentedgenerationlostinthemiddleopen-domainquestionansweringdecoder-onlylanguagemodelparallelcontextencodingKV-Fusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Retrieval-augmented question answering fails when the one passage holding the answer sits in the middle of a long context. This paper tries to establish that this 'lost in the middle' failure can be trained away for decoder-only models by changing how retrieved passages are presented: each passage is encoded separately with the same positional slots, and a trainable decoder reads the fused key-value caches instead of a sequential token stream. On NQ, TriviaQA, and POPQA, the resulting models keep accuracy essentially constant as the gold passage moves, while base Llama3 models droop in the middle and drop sharply when contexts are shuffled; reported gains over baseline models in shuffled settings are 21.4%, 6.4%, and 6.6%. If the explanation is right, RAG pipelines could tolerate many irrelevant passages and skip reranking without losing accuracy.

What carries the argument

The load-bearing object is the reshaped key-value cache. Per layer, the caches from all N passages are concatenated along the token axis, giving one cache block per layer that is injected into the trainable decoder's attention. This is what enforces position invariance: passage tokens always occupy positions 1 through n, and the instruction and answer tokens occupy positions n+1 onward, so reordering the passages does not alter any positional embedding seen by the decoder.

What would settle it

Fine-tune a standard Llama3-8B on the same training instances used for KV-Fusion (one gold passage, 19 negative passages, and the evidence string in the target) with ordinary sequential encoding, then compare shuffled-context exact match and token-level match; if the standard model also reaches near-perfect token-level match, the cache fusion is not the cause.

Watch

Extended reading notes

Core claim

The paper's central assertion is that decoder-only language models can be made context-order invariant for RAG by conditioning answer generation on fused key-value caches instead of on a linearly ordered prompt. A frozen prefill decoder reads each retrieved passage in parallel, so every passage is assigned the same local positions (1 through n); the layer-wise key and value caches are concatenated along the token axis and fed into a trainable decoder whose own query and answer tokens start at position n+1. Because a passage never changes its position when the passage list is shuffled, the model has no positional signal by which to behave differently, and the paper reports more than 99% token-level match between first-position and shuffled presentations, compared with low single-digit-to-teens match for standard Llama3 and Llama3.1.

Load-bearing premise

The comparison assumes that fine-tuning on gold answers plus extracted evidence is not itself what produces the stable, order-insensitive behavior, because the KV-Fusion models receive that fine-tuning while Llama3, REPLUG, and PAM-QA do not.

Editorial extensions

If this is right

  • Reranking becomes optional: KV-Fusion evaluated on the unranked top-40 passages matches or beats Llama3 and Llama3.1 evaluated on reranked top-20 passages on NQ, TriviaQA, and POPQA.
  • Accuracy stops degrading as context grows: KV-Fusion stays within a small margin between top-5 and top-40 passages, whereas PAM-QA loses roughly half its accuracy on average between top-20 and top-40.
  • Outputs become reproducible under shuffling: with greedy decoding, shuffled contexts produce token-identical answers in over 99% of cases, so downstream use of RAG outputs no longer depends on retrieval order.
  • The effect transfers across base models: the same training recipe gives position-invariant readers from both Llama3-8B and Llama3.1-8B.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not tested in the paper: because the prefill decoder's caches depend only on the passages, the same fused cache could in principle serve many different questions about the same document set, which would make repeated RAG calls cheaper.
  • Not tested in the paper: the invariance is demonstrated for single-hop QA; whether cross-passage composition survives in multi-hop datasets like HotpotQA or MuSiQue is a direct consequence the authors flag as open.
  • If the mechanism is the cause rather than the fine-tuning data, the same cache-reshaping trick could be applied to continued pretraining, making order invariance a property of the base model rather than a per-task fine-tune.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes KV-Fusion, a two-decoder architecture for decoder-only RAG readers. A frozen prefill decoder encodes each retrieved passage independently with identical local positional embeddings and produces per-passage KV caches that are concatenated along the token dimension; a trainable decoder then generates answers conditioned on these fused caches plus the question, using next-token prediction. The design ensures that shuffling the order of passages does not change the attention geometry, thus yielding position-invariant outputs by construction. Experiments on NQ, TriviaQA, and POPQA report EM accuracy, token-level match, robustness to more passages, and comparisons to REPLUG and PAM QA, claiming accuracy gains of 21.4, 6.4, and 6.6 percentage points in shuffled settings.

Significance. The architectural insight that identical local positions plus cache fusion without cross-passage attention yields exact order invariance is clean and should hold. The TLM metric is a sensible way to measure token-level consistency. However, the quantitative claims that KV-Fusion improves accuracy are not yet separable from the effects of supervised fine-tuning and the evidence-augmented target format, so the contribution's empirical value is unproven. If a properly controlled comparison confirms the gains, this would be a practical contribution to position-invariant RAG.

major comments (4)
  1. [§4, Table 1 and Figure 4] The KV models are fine-tuned on each dataset with gold answers and GPT-4o-extracted evidence, while Llama3, REPLUG, and PAM QA are evaluated zero-shot or with default settings. The reported accuracy differences (21.4%, 6.4%, and 6.6% in shuffled settings) could therefore be due entirely to supervised fine-tuning rather than to the cache-fusion mechanism. Please include a control reader fine-tuned on the same training data and target format (answer plus evidence) with standard sequential cross-attention over all passages, and compare it against KV-Fusion trained on the same data. Additionally, ablate the evidence target by training KV-Fusion without the evidence in the target to isolate its effect.
  2. [§2, §5, and §7] The near-perfect TLM reported in Table 2 is structurally guaranteed by the architecture: each passage is encoded at identical local positions and the reshaped cache is a permutation-invariant multiset of key-value pairs, so the output distribution is exactly unchanged under passage shuffling (modulo floating-point ordering effects). The conclusion's statement that KV-Fusion 'trains language models to be context-order invariant' overstates the role of training; the invariance is a property of the inference-time construction. Please revise the framing to make clear that the empirical TLM result confirms the architecture rather than a learned behavior.
  3. [Appendix A.5 and §4] The input preprocessing differs in a way that may confound the comparison. For KV-Fusion, each passage is tokenized to a maximum length of 192 tokens with left truncation (removing tokens from the beginning of long passages), as stated in Appendix A.5. For the baselines, the paper does not report whether the same per-passage truncation is applied or whether the entire concatenated context is truncated at the sequence level. If baselines receive longer or differently truncated contexts, the accuracy gap may reflect input processing choices rather than the fusion mechanism. Please specify the exact input construction for all systems and ensure that the only difference between conditions is the attention/fusion mechanism.
  4. [§5, Table 3] The claim that KV-Fusion 'eliminates the need for ranking in the RAG pipeline' is based on a comparison where KV models use the original top-40 passages while baselines use the top-20 reranked passages, and the KV models are fine-tuned. This comparison mixes two variables (passage count and fine-tuning) and does not isolate the fusion mechanism. A fair comparison would evaluate all systems on the same number of passages (e.g., top-20 reranked) and with the same fine-tuning status, or at least include an SFT baseline on concatenated contexts. Also, no variance or significance testing is reported for any of the differences in Tables 1 and 3, so the stability of the gains is unclear.
minor comments (6)
  1. [Abstract] The word 'irrelevent' should be 'irrelevant'.
  2. [§3.2] The word 'preprend' should be 'prepend'.
  3. [Appendix A.2.1] The prompt template contains a typo: 'inlcude' should be 'include'.
  4. [Table 3 caption] The capitalization of 'LLama3' is inconsistent with the rest of the paper; use 'Llama3' consistently.
  5. [§6] The relationship to parallel context encoding (Yen et al., 2024) and Fusion-in-Decoder (Izacard and Grave, 2020) should be explicitly discussed to position the novelty of KV-Fusion relative to these prior works.
  6. [§5] In the TLM definition, the notation p and s is reused without explicit definition; please clarify that p and s are the full generated responses in the POS1 and Shuffled scenarios.

Circularity Check

2 steps flagged · score 7.0 of 10

Position invariance is guaranteed by the architecture rather than discovered, and the Table 1 accuracy gains are confounded with supervised fine-tuning on gold answers and evidence.

  1. self definitional [Section 2 (Method), Prefill Decoder / Trainable Decoder; reported in Section 4, Figure 3, and Table 2]
    "Prefill Decoder ( Dp) extracts the KV-caches from multiple input passages in parallel, resulting in the injection of identical local positional embeddings { ⃗ p1, ⃗ p2, . . . , ⃗ pn}. ... We then train Dt using next-token prediction, conditioning on the reshaped KV-caches rather than previous tokens: Dt(y|q, C′) ≜ Dt(y|q, {Kl, Vl}L l=1)"

    Because every passage is encoded by the frozen Dp at the same local positions 1..n and the fused cache is the concatenation of the resulting (key, value) pairs, shuffling C only permutes the token axis of the cache. Dt's attention over these keys is permutation-invariant: attention weights depend on q·k and outputs on weighted sums of v, neither of which changes when the passage order is changed (up to floating-point summation order). Therefore the near-100% Token-Level Match in Table 2 and the flat accuracy curves in Figure 3 follow from the construction by definition; they are not empirical discoveries about training.

  2. fitted input called prediction [Section 3.1 (Dataset Construction), Section 3.2 (Training), Section 4 Table 1 and Figure 4]
    "Each training instance consists of one gold context and 19 negative contexts. ... Finally, we add answer tokens, which contain both an answer string and a key phrase as evidence, as described in Section 3.1. ... We fine-tune each dataset with a maximum learning rate of 2 × 10−5 using the AdamW. ... using default settings for PAM QA and the same configurations as Llama3 for RE-PLUG."

    The reported accuracy gains of KV-Llama3 over Llama3, REPLUG, and PAM-QA (21.4%, 6.4%, and 6.6% in the shuffled setting) are presented as evidence for the fusion mechanism, but the KV models are supervised on the gold answer and an extracted evidence phrase in the exact target format for each dataset, while the comparison systems are used zero-shot or with default settings. A standard fine-tuned reader trained on the same gold-answer/evidence targets would be expected to show similar gains, so the superiority is not attributable to position-invariant cache fusion. The fitted target (the gold answer string) is effectively renamed as a prediction of the method; the comparison does not isolate the fusion contribution.

full rationale

The strongest circularity is in the headline property itself. The architecture assigns identical local positional embeddings to every passage in the frozen prefill decoder and then conditions the trainable decoder only on the reshaped cache; this makes the output distribution invariant to passage order by construction. The paper nevertheless presents position invariance and the ~99.6% token-level matches as empirical results, and the conclusion credits training with producing the invariance. The second, separate issue is that the accuracy/robustness comparisons in Table 1 and Figure 4 do not hold the training procedure constant: KV models are fine-tuned with gold answers and GPT-4o-extracted evidence, while Llama3, REPLUG, and PAM-QA are not. Thus the headline '21.4% / 6.4% / 6.6%' gains could be produced by ordinary supervised fine-tuning, independent of the fusion construction. I found no load-bearing self-citation: the paper's own prior work (Oh and Thorne 2023) is cited only as related work on detrimental contexts, not as a uniqueness theorem or as justification for the architecture. The derivation of the KV reshaping itself is explicit and self-contained; the circularity lies in claiming as discovered a property that is definitionally guaranteed, and in attributing accuracy gains to fusion when the experimental comparison is confounded. Score 7 reflects partial circularity: one central claim reduces by construction, and the other central empirical claim is not isolated from the fitted supervision.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim of order invariance is mostly carried by the architectural choice of identical local positions and cache fusion; the only fitted or chosen quantities are training hyperparameters and filtering rules. No new physical or parametric entity is introduced.

free parameters (5)
  • training context count N=20 (1 gold plus 19 negatives)
    Chosen by hand; position-invariance holds for any N, but accuracy and robustness at top-5, top-10, and top-40 may depend on the training-time passage count.
  • per-passage token length n=192 with left truncation
    Chosen by hand; target token positions start at p_{n+1}, so the method's positional alignment depends on this fixed length, and long passages lose their beginnings.
  • target token max length 192
    Chosen for training; limits the combined length of instruction, question, answer, and evidence tokens.
  • fine-tuning hyperparameters: lr=2e-5, epochs=2, batch size=64
    Standard fine-tuning choices; no sensitivity analysis is reported, so the empirical results could depend on these values.
  • GPT-4o evidence acceptance rule
    Instances where GPT-4o returns no positive evidence are discarded; this changes the dataset composition and may bias training and evaluation.
assumptions (5)
  • standard math Attention scores depend only on content and relative position, so identical local positions across passages make the fused cache permutation-invariant.
    The paper relies on this property in Section 2 without proving it; it is the reason shuffling passages does not change outputs.
  • domain assumption Key-value caches extracted by the frozen prefill decoder remain compatible with the trainable decoder after its parameters are updated.
    Dt is initialized from the same base model but fine-tuned; the paper assumes the cached keys and values remain usable, as used in Algorithm 1.
  • domain assumption The trainable decoder learns to answer from the fused caches and query without relying on seeing the gold answer during inference.
    The training template includes the answer and evidence tokens, but the inference-time input format is not specified, so the assumption is implicit and untested.
  • domain assumption DPR and Contriever retrieve the gold context within the evaluated top-k lists.
    The whole pipeline depends on retriever recall; no recall analysis or oracle-retrieval experiment is provided.
  • ad hoc to paper GPT-4o-extracted evidence is reliable and the NONE-based filtering does not distort benchmark comparability.
    Dataset construction in Section 3.1 and Appendix A.2 filters data based on GPT-4o outputs; this is not independently validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Parallel Key-Value Cache Fusion for Position Invariant RAG." pith.science (2026). https://pith.science/paper/KDVCOVHJ

@misc{pith2026250107523,
  author       = {Pith},
  title        = {Pith review of: Parallel Key-Value Cache Fusion for Position Invariant RAG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KDVCOVHJ}},
  note         = {Machine review of arXiv:2501.07523}
}
read the original abstract

Recent advancements in Large Language Models (LLMs) underscore the necessity of Retrieval Augmented Generation (RAG) to leverage external information. However, LLMs are sensitive to the position of relevant information within contexts and tend to generate incorrect responses when such information is placed in the middle, known as `Lost in the Middle' phenomenon. In this paper, we introduce a framework that generates consistent outputs for decoder-only models, irrespective of the input context order. Experimental results for three open domain question answering tasks demonstrate position invariance, where the model is not sensitive to input context order, and superior robustness to irrelevent passages compared to prevailing approaches for RAG pipelines.

Figures

Figures reproduced from arXiv: 2501.07523 by the authors.

Figure 1
Figure 1. Illustration of the KV-Fusion model: Gener [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of KV-Fusion Architecture. Dp denotes Prefill decoder and Dt represents Trainable decoder. We employ the off-the-shelf LLM to extract the key and value states of the retrieved contexts independently. Then reshaping these caches to train the LLM with task instructions along with questions to generate answers. and value heads |H |, each with a dh dimension. We denote the trainable decoder as Dt , and represen… view at source ↗
Figure 3
Figure 3. Comparison of EM Accuracy between KV-Llama3 and Llama3 across different gold context positions. KV-Llama3 maintains its accuracy, while Llama3 shows a tendency for the ‘lost in the middle’ problem. (a) NQ (b) TQA (c) POPQA [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Accuracies of baseline and KV models in two scenarios: 1) [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: Comparison of EM Accuracy between KV-Llama3.1 and Llama3.1 across different gold context positions. With varying gold context positions, KV-Llama3.1 illustrates consistent accuracies across datasets. However, Llama3.1 suffers from the ‘Lost in the middle’ problem, whic…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 6 canonical work pages

  1. [1]

    Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, and Jian-Guang Lou. 2024. https://arxiv.org/abs/2404.16811 Make your llm fully utilize the context . Preprint, arXiv:2404.16811

    arXiv 2024

  2. [2]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. https://arxiv.org/abs/2310.11511 Self-rag: Learning to retrieve, generate, and critique through self-reflection . Preprint, arXiv:2310.11511

    arXiv 2023

  3. [3]

    Rae, Erich Elsen, and Laurent Sifre

    Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Ori...

    arXiv 2022

  4. [4]

    Brian J Chan, Chao-Ting Chen, Jui-Hung Cheng, and Hen-Hsen Huang. 2024. https://arxiv.org/abs/2412.15605 Don't do rag: When cache-augmented generation is all you need for knowledge tasks . Preprint, arXiv:2412.15605

    arXiv 2024

  5. [5]

    Abhimanyu Dubey, others Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of mod...

    arXiv 2024

  6. [6]

    Feiteng Fang, Yuelin Bai, Shiwen Ni, Min Yang, Xiaojun Chen, and Ruifeng Xu. 2024. https://doi.org/10.18653/v1/2024.acl-long.540 Enhancing noise robustness of retrieval-augmented language models with adaptive adversarial training . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 100...

    doi:10.18653/v1/2024.acl-long.540 2024

  7. [7]

    Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Ankita Naik, Pengshan Cai, and Alfio Gliozzo. 2022. https://doi.org/10.18653/v1/2022.naacl-main.194 R e2 G : Retrieve, rerank, generate . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2701...

    doi:10.18653/v1/2022.naacl-main.194 2022

  8. [8]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. https://arxiv.org/abs/2002.08909 Realm: Retrieval-augmented language model pre-training . Preprint, arXiv:2002.08909

    arXiv 2020

Show all 51 references
  1. [9]

    Junqing He, Kunhao Pan, Xiaoqun Dong, Zhuoyang Song, LiuYiBo LiuYiBo, Qianguosun Qianguosun, Yuxin Liang, Hao Wang, Enming Zhang, and Jiaxing Zhang. 2024. https://doi.org/10.18653/v1/2024.acl-long.736 Never lost in the middle: Mastering long-context question answering with pos...

    2024 doi

  2. [10]

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024 a . Ruler: What's the real context size of your long-context language models? arXiv preprint arXiv:2404.06654

    2024 arXiv

  3. [11]

    Cheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang, Long Le, Abhishek Kumar, James Glass, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, and Tomas Pfister. 2024 b . https://doi.org/10.18653/v1/2024.findings-acl.890 Found in the middle: Calibrating positional attentio...

    2024 doi

  4. [13]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023 b . https://arxiv.org/abs/2311.05232 A survey on hallucination in large language models: Principles, taxonomy, challenges, a...

    2023 arXiv

  5. [14]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. https://doi.org/10.48550/ARXIV.2112.09118 Unsupervised dense information retrieval with contrastive learning

    2021 doi arXiv

  6. [15]

    Gautier Izacard and Edouard Grave. 2020. https://arxiv.org/abs/2007.0128 Leveraging passage retrieval with generative models for open domain question answering . arXiv preprint

    2020

  7. [16]

    Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. https://arxiv.org/abs/2208.03299 Atlas: Few-shot learning with retrieval augmented language models . Preprint, a...

    2022 arXiv

  8. [17]

    Chao Jin, Zili Zhang, Xuanlin Jiang, Fangyue Liu, Xin Liu, Xuanzhe Liu, and Xin Jin. 2024. https://arxiv.org/abs/2404.12457 Ragcache: Efficient knowledge caching for retrieval-augmented generation . Preprint, arXiv:2404.12457

    2024 arXiv

  9. [18]

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational...

    2017 doi

  10. [19]

    Gregory Kamradt. 2023. https://github.com/gkamradt/LLMTest_NeedleInAHaystack/tree/main Needle in a haystack - pressure testing llms

    2023

  11. [20]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on Empiric...

    2020 doi

  12. [21]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019 doi

  13. [22]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. https://arxiv.org/abs/2005.11401 Retrieval-augmented generation for knowledge-int...

    2021 arXiv

  14. [23]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. https://api.semanticscholar.org/CorpusID:259360665 Lost in the middle: How language models use long contexts . Transactions of the Association for Computational L...

    2023

  15. [24]

    Songshuo Lu, Hua Wang, Yutian Rong, Zhi Chen, and Yaohua Tang. 2024. https://arxiv.org/abs/2410.07590 Turborag: Accelerating retrieval-augmented generation with precomputed kv caches for chunked text . Preprint, arXiv:2410.07590

    2024 arXiv

  16. [25]

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. https://doi.org/10.1145/3626772.3657951 Fine-tuning llama for multi-stage text retrieval . In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGI...

    2024

  17. [26]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://arxiv.org/abs/2212.10511 When not to trust language models: Investigating effectiveness of parametric and non-parametric memories . Preprint, arXiv:2212.10511

    2023 arXiv

  18. [27]

    Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, and Nat McAleese. 2022. https://arxiv.org/abs/2203.11147 Teaching language models to support answers with verifie...

    2022 arXiv

  19. [28]

    Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.63 Document ranking with a pretrained sequence-to-sequence model . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 708--718, Onl...

    2020 doi

  20. [29]

    Philhoon Oh and James Thorne. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.776 Detrimental contexts in open-domain question answering . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11589--11605, Singapore. Association for Computation...

    2023 doi

  21. [30]

    OpenAI. 2024. https://openai.com/index/hello-gpt-4o/ Openai. 2024. gpt-4o

    2024

  22. [31]

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch

    2017

  23. [32]

    Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rockt \"a schel, and Sebastian Riedel. 2021. https://doi.org/10.18653/v1/2021.naacl-main.200 KIL...

    2021 doi

  24. [33]

    Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong. 2021. https://arxiv.org/abs/2103.11251 Interpretable machine learning: Fundamental principles and 10 grand challenges . Preprint, arXiv:2103.11251

    2021 arXiv

  25. [34]

    Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. 2023. https://arxiv.org/abs/2302.00093 Large language models can be easily distracted by irrelevant context . Preprint, arXiv:2302.00093

    2023 arXiv

  26. [35]

    Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. 2024 a . https://doi.org/10.18653/v1/2024.naacl-short.69 Trusting your evidence: Hallucinate less with context-aware decoding . In Proceedings of the 2024 Conference of the North America...

    2024 doi

  27. [36]

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2024 b . https://doi.org/10.18653/v1/2024.naacl-long.463 REPLUG : Retrieval-augmented black-box language models . In Proceedings of the 2024 Conference of the ...

    2024 doi

  28. [37]

    Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, and Furu Wei. 2024. https://arxiv.org/abs/2405.05254 You only cache once: Decoder-decoder architectures for language models . Preprint, arXiv:2405.05254

    2024 arXiv

  29. [38]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, et al. 2024. https://arxiv.org/abs/2403.05530 Gemini 1.5: Unlocking multimodal understanding across millio...

    2024 arXiv

  30. [39]

    Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Q...

    2022 arXiv

  31. [40]

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. https://doi.org/10.18653/v1/N18-1074 FEVER : a large-scale dataset for fact extraction and VER ification . In Proceedings of the 2018 Conference of the North A merican Chapter of the Associatio...

    2018 doi

  32. [41]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. https://doi.org/10.1162/tacl_a_00475 Musique: Multihop questions via single-hop question composition . Transactions of the Association for Computational Linguistics, 10:539--554

    2022 doi

  33. [42]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 arXiv

  34. [43]

    Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2023. https://arxiv.org/abs/2310.04408 Recomp: Improving retrieval-augmented lms with compression and selective augmentation . Preprint, arXiv:2310.04408

    2023 arXiv

  35. [44]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA : A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing ( EMNLP )

    2018

  36. [45]

    Howard Yen, Tianyu Gao, and Danqi Chen. 2024. https://arxiv.org/abs/2402.16617 Long-context language modeling with parallel context encoding . Preprint, arXiv:2402.16617

    2024 arXiv

  37. [46]

    Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2024 a . https://openreview.net/forum?id=ZS4m74kZpH Making retrieval-augmented language models robust to irrelevant context . In The Twelfth International Conference on Learning Representations

    2024

  38. [47]

    Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2024 b . https://arxiv.org/abs/2310.01558 Making retrieval-augmented language models robust to irrelevant context . Preprint, arXiv:2310.01558

    2024 arXiv

  39. [48]

    Yijiong Yu, Huiqiang Jiang, Xufang Luo, Qianhui Wu, Chin-Yew Lin, Dongsheng Li, Yuqing Yang, Yongfeng Huang, and Lili Qiu. 2024 a . https://arxiv.org/abs/2406.02536 Mitigate position bias in large language models via scaling a single dimension . Preprint, arXiv:2406.02536

    2024 arXiv

  40. [49]

    Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. 2024 b . https://openreview.net/forum?id=S1fc92uemC Rank RAG : Unifying context ranking with retrieval-augmented generation in LLM s . In The Thirty-eighth Annual Conferenc...

    2024

  41. [50]

    Honglei Zhuang, Zhen Qin, Rolf Jagerman, Kai Hui, Ji Ma, Jing Lu, Jianmo Ni, Xuanhui Wang, and Michael Bendersky. 2023. https://doi.org/10.1145/3539618.3592047 Rankt5: Fine-tuning t5 for text ranking with ranking losses . In Proceedings of the 46th International ACM SIGIR Conf...

    2023

  42. [51]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  43. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.