Pith. sign in

REVIEW 3 major objections 5 minor 75 references

TracLLM: A Generic Framework for Attributing Long Context LLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TracLLM pinpoints the texts that steer a long-context LLM's output.

desk verdict TracLLM's hierarchical pruning is a real advance for long-context attribution, with strong joint-attack results, but the theory overpromises relative to the actual method. read the letter →

arxiv 2506.04202 v3 pith:YKAIHCEF submitted 2025-06-04 cs.CR cs.AIcs.LG

Yanting Wang , Wei Zou , Runpeng Geng , Jinyuan Jia This is my paper
classification cs.CRcs.AIcs.LG
keywords contexttracebacklong-contextLLMattributionShapleyvaluepromptinjectionforensicsknowledgecorruptionattacksretrieval-augmentedgenerationmodelexplainabilityinformedsearch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long-context LLMs answer questions using hundreds of passages, so when an answer is wrong or malicious it is hard to say which passage caused it. This paper introduces TracLLM, a framework that treats each passage as a text in a cooperative game and searches the context for the top-K passages that most raise the probability of the observed output. The search is a divide-and-conquer: score groups of passages, keep the K best-scoring groups, split them, and repeat until single passages remain. The authors claim this makes Shapley-scale attribution practical on long contexts, and that denoising and ensembling the scores makes the attribution accurate enough to serve as forensic evidence after attacks.

What carries the argument

The load-bearing object is the informed search tree over groups of texts. At each level the context is partitioned into about 2K groups, each group is scored by the chosen attribution method (usually Shapley with Monte-Carlo permutations, using the conditional probability of the output, or BLEU score for black-box LLMs, as the value function), the K groups with the largest scores survive, and the search descends only into those groups. Two refinements carry the accuracy: contribution score denoising, which averages only the top β fraction of marginal contributions rather than all permutations, and contribution score ensembling, which takes the maximum score over several attribution methods. A formal guarantee (Proposition 1) shows that under unanimity or existence games with an indicator value function and K at least the number of decisive texts, the decisive texts are never pruned.

What would settle it

Build a context of about 200 passages with a pair of texts that only produce the target output when both are present, place them in two different first-level groups, and run TracLLM with K=5 and a small number of Shapley permutations using the conditional-probability value function; if one group scores below the top-K and is pruned before the texts are ever combined, recall drops and the pair is missed. Running the same case with the indicator value function should recover them, isolating the value-function assumption as the difference.

Watch

Extended reading notes

Core claim

The paper's central claim is that context traceback for long-context LLMs can be made both effective and efficient by informed search over groups of texts rather than by attributing each text independently. TracLLM repeatedly scores groups of passages with any perturbation-based attribution method, such as Shapley, leave-one-out, or single-text contribution, prunes all but the K most contributory groups, and splits the survivors until the top-K individual texts remain. With a limited budget of Shapley permutations this finds the texts that lead to an output, and the authors report that removing the identified texts reduces attack success rate to near zero across prompt injection, knowledge corruption, and agent-memory backdoor attacks, with 89 percent recall on PoisonedRAG over NQ.

Load-bearing premise

The pruning step assumes that a group's contribution score, computed with a limited number of Shapley permutations on the conditional probability (or BLEU) value function, is a reliable signal for whether the group contains a contributing text; if a group containing an important text scores outside the top-K, that text is permanently discarded, and the formal guarantee only covers binary indicator value functions under strict unanimity or existence games.

Editorial extensions

If this is right

  • Forensic analysis after prompt injection or RAG poisoning can be automated: given a wrong output, TracLLM returns passages whose removal makes the attack fail, with attack success rate near zero in the reported evaluations.
  • Debugging LLM-based systems becomes tractable: in the paper's case study, the framework isolated the joke comment that produced a misleading glue-in-the-sauce answer.
  • Attribution cost scales as O(K·e·log n) queries rather than O(e·n), so long contexts are feasible when K is small.
  • Because the search descends only into top-scoring groups, the framework works with black-box LLMs by swapping the value function from conditional probability to BLEU similarity.
  • Supporting-evidence tracing is a direct corollary: 77 percent of texts TracLLM found for Natural Questions answers were judged to support the answer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the formal guarantee relies on a binary indicator value function, the practical accuracy rests on an empirical link: probability-based and BLEU-based scores must preserve the ranking that the indicator function would give. A testable extension is to benchmark TracLLM on cases where decisive texts contribute only jointly across groups.
  • The divide-and-prune search could be reused as a cheap screening step inside other attribution pipelines: run TracLLM to narrow the context, then apply a more expensive method such as larger-permutation Shapley to the surviving passages.
  • An attacker wanting to evade traceback would need to spread influence across many passages so that no single group carries a distinguishing score; the paper's attack-success results suggest current attacks do not do this, but constructing such distributed injections is a natural stress test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces TracLLM, a framework for context traceback: given a long context, an instruction, and an LLM output, it identifies the K texts most responsible for the output. The method recursively partitions the context into groups, scores each group with a perturbation-based attribution method (STC, LOO, or Shapley), retains the top-K groups, and repeats until single texts remain. Two accuracy enhancements are proposed: top-β denoising of Shapley marginal contributions and a max-ensemble over attribution methods. The paper proves a lossless-pruning guarantee under a unanimity/existence game with an indicator value function, and evaluates the method on prompt injection, knowledge corruption, and agent attacks across several LLMs, reporting high recall and large reductions in attack success rate after removing identified texts.

Significance. If the empirical results generalize, TracLLM is a practical forensic tool for long-context LLMs and an efficiency improvement over full Shapley. The paper's strengths are its broad evaluation (6 datasets, 6 LLMs, 13 attacks), clear ablations, and the release of code and data. The central efficiency claim rests on pruning being lossless; the theoretical support for this is limited to an idealized value function, which is the main weakness. The experiments are extensive and the results are plausible, but the mismatch between the formal guarantee and the actual scoring regime leaves a load-bearing gap.

major comments (3)
  1. [§3.3, Appendix B] Proposition 1 assumes the value function is the binary indicator I(f(I⊕U)=O), exact Shapley values, and that generation is exactly a unanimity or existence game. In contrast, the experiments (§4.1) use conditional probability p_f(O|I⊕U) or BLEU scores, estimate Shapley from 5–20 Monte-Carlo permutations, and then apply β-denoising and max-ensemble (§3.2). None of these regimes satisfies the theorem's assumptions, so the proof that pruning is lossless does not apply to the actual scoring used in the paper. This matters because the joint-attack scenario in Table 3, where pruning is most stressed, is exactly the case not covered by Proposition 1. The authors should either extend the theory to continuous value functions with sampling error, or substantially soften the claims in Section 5 that TracLLM "provably" identifies malicious texts and is non-trivial to evade.
  2. [Table 3, §4.2] The paper's key advantage over STC is said to be handling multiple texts that jointly produce an output, but the supporting experiments are reported without dataset size, number of samples, or variance. In particular, Table 3 shows TracLLM precision of 0.43 for prompt injection and 0.36 for knowledge corruption, meaning many false positives; with K=5 and only two malicious texts, a recall of 0.95 is possible even if pruning is imperfect. Additional experiments with more joint-attack cases and an analysis of pruning failures across iterations are needed to support the central claim.
  3. [Table 6, §4.2] The ASR-after-removal metric is reported with K=5 while only three malicious texts are injected. Removing five texts can reduce ASR even if the method's precision is moderate; precision/recall are the more direct metrics. The authors should report ASRa conditioned on cases where all truly malicious texts are among the removed K texts, or adjust K to the number of injected texts, to make the forensic claim interpretable.
minor comments (5)
  1. [§4.1, Table 14] The experimental setup lists the LLMs tested, but GLM-4-9B-Chat and Gemma-3-1B appear in Table 14 without being mentioned in the setup; please add them to the list or clarify.
  2. [§4.2, Figure 4] The text contains a typo: "Shaply" should be "Shapley" in the sentence about computation cost at 40,000 words.
  3. [§2.2.1, reference [47]] STC is cited as RISE (Randomized Input Sampling for Explanation), but RISE is a different method; please provide an appropriate reference for the single-text conditional-probability baseline or rename the baseline.
  4. [§4.3, Figure 5] Figure 5 shows the effect of β on precision and recall, but no error bars or multiple seeds are reported; adding them would strengthen the stability claim about the default β=20%.
  5. [Algorithm 1] The loop at lines 6–9 divides until the number of groups exceeds K; for K=1 this means the algorithm always begins with two groups, which is correct but could use a brief comment for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TracLLM's formal guarantee is proved from explicit game-theoretic assumptions, and its empirical claims are validated against external ground-truth attack texts rather than against its own scoring definitions.

full rationale

TracLLM's derivation chain is self-contained and does not reduce to its inputs. The only formal guarantee, Proposition 1, is proved from Definitions 1 and 2 (unanimity/existence games) using exact Shapley with the indicator value function; it is a mathematical consequence of those definitions and standard Shapley properties, not a fitted parameter renamed as a prediction. The experimental evaluations use ground-truth injected malicious texts and measure precision/recall and attack-success rates before and after removing the predicted texts, so the empirical claims are validated against external attack outcomes rather than against the framework's own scoring procedure. The self-citations that appear ([36] for a combined prompt-injection attack, [74] for PoisonedRAG) are used as attack benchmarks with open-source implementations, not as load-bearing support for TracLLM's contribution or its theoretical claims. The fact that Proposition 1's assumptions (binary indicator values, unanimity/existence structure, exact Shapley) are narrower than the experimental protocol (probability/BLEU value functions, Monte-Carlo Shapley estimates, top-beta denoising, and max-ensemble) is a limitation on the transfer of the formal guarantee, but it is not a circularity: the experiments are not constructed so that their outputs equal the scores that define them.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The framework rests on the reliability of group-level attribution scores for pruning and on order-insensitive value functions; these are domain assumptions about LLM and attribution behavior, only partially supported by the idealized theory.

free parameters (4)
  • K (number of texts returned) = 5
    Hyperparameter chosen by the authors; ablations show precision/recall tradeoff (Figure 7).
  • beta (denoising fraction) = 0.20
    Fraction of largest marginal contributions averaged; set to 20% for stability, though 5% gave slightly better results (Figure 5).
  • w (LOO scaling weight) = 2
    Scaling factor for LOO scores in the ensemble; default set to 2, insensitive across datasets (Figure 8).
  • number of Shapley permutations (ensemble) = 20
    Permutations used for the Shapley component of the ensemble; cost-efficiency tradeoff (Table 4).
assumptions (3)
  • ad hoc to paper A group containing a contributing text will receive a higher contribution score than a group without one, so pruning the lowest-scoring groups is safe.
    Algorithm 1 Step II permanently discards pruned groups; Proposition 1 proves this only for indicator value functions and strict unanimity/existence games, not for the conditional probability or BLEU value functions used in experiments.
  • domain assumption The value function v(R) = p_f(O|I x R) is invariant to the order in which texts are concatenated, so Shapley permutations are well-defined.
    Section 2.1 defines context as a fixed concatenation; Shapley sampling permutes groups, but order sensitivity of LLM outputs is not examined.
  • domain assumption Monte Carlo Shapley with a small number of permutations approximates the true Shapley values sufficiently for group ranking.
    The algorithm relies on 20 permutations for the ensemble; no convergence analysis is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TracLLM: A Generic Framework for Attributing Long Context LLMs." pith.science (2026). https://pith.science/paper/YKAIHCEF

@misc{pith2026250604202,
  author       = {Pith},
  title        = {Pith review of: TracLLM: A Generic Framework for Attributing Long Context LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YKAIHCEF}},
  note         = {Machine review of arXiv:2506.04202}
}
read the original abstract

Long context large language models (LLMs) are deployed in many real-world applications such as RAG, agent, and broad LLM-integrated applications. Given an instruction and a long context (e.g., documents, PDF files, webpages), a long context LLM can generate an output grounded in the provided context, aiming to provide more accurate, up-to-date, and verifiable outputs while reducing hallucinations and unsupported claims. This raises a research question: how to pinpoint the texts (e.g., sentences, passages, or paragraphs) in the context that contribute most to or are responsible for the generated output by an LLM? This process, which we call context traceback, has various real-world applications, such as 1) debugging LLM-based systems, 2) conducting post-attack forensic analysis for attacks (e.g., prompt injection attack, knowledge corruption attacks) to an LLM, and 3) highlighting knowledge sources to enhance the trust of users towards outputs generated by LLMs. When applied to context traceback for long context LLMs, existing feature attribution methods such as Shapley have sub-optimal performance and/or incur a large computational cost. In this work, we develop TracLLM, the first generic context traceback framework tailored to long context LLMs. Our framework can improve the effectiveness and efficiency of existing feature attribution methods. To improve the efficiency, we develop an informed search based algorithm in TracLLM. We also develop contribution score ensemble/denoising techniques to improve the accuracy of TracLLM. Our evaluation results show TracLLM can effectively identify texts in a long context that lead to the output of an LLM. Our code and data are at: https://github.com/Wang-Yanting/TracLLM.

Figures

Figures reproduced from arXiv: 2506.04202 by the authors.

Figure 1
Figure 1. Visualization of context traceback. based systems—such as Retrieval-Augmented Generation (RAG) [30, 34], agents [1, 60, 69], and many LLM-integrated applications—to incorporate a broader range of external in￾formation for solving complex real-world tasks. For example, a long-context LLM enables: 1) RAG systems like Bing Copi￾lot [2], Google Search with AI Overviews [3], and Perplexity AI [8] to leverage a large numb… view at source ↗
Figure 2
Figure 2. Overview of TracLLM. Given an instruction, an output, an LLM, and a long context containing a set of texts, TracLLM searches T2 and T6 from the context that induce an LLM to generate Pwned! 3.1 A Generic Context Traceback Framework We iteratively search for top-K texts in the context T = {T1,T2,··· ,Tn} contributing to the output O of an LLM. To this end, we start by recursively dividing texts in T into smaller grou… view at source ↗
Figure 3
Figure 3. Comparing the precision and recall of TracLLM with Shapley and LIME/Context-Cite under different computation costs. the recall of TracLLM increases when an attacker injects less number of malicious instructions/texts. We suspect the reason is that TracLLM considers the influence of each text when combined with other texts, allowing it to more effec￾tively isolate and identify malicious instructions/texts when their … view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Comparing the computation costs of TracLLM [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Impact of β on contribution score denoising. In summary, TracLLM can effectively find malicious texts crafted by diverse attacks that induce an LLM to generate attacker-desired outputs. The effectiveness of TracLLM under a large number of malicious texts: TracLLM can i…
Figure 7
Figure 7. Figure 7: Impact of K on TracLLM. D Evaluation for Other Applications D.1 Debugging LLM-based Systems Suppose a long context LLM generates a misleading answer based on a long context. TracLLM can be used to identify texts responsible for the misleading answer. Experimental setup…
Figure 8
Figure 8. Figure 8: Impact of w on TracLLM [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: The output of Google Search with AI Overviews [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

75 extracted references · 55 canonical work pages

  1. [1]

    https: //github.com/Significant-Gravitas/AutoGPT

    AutoGPT: Build, Deploy, and Run AI Agents. https: //github.com/Significant-Gravitas/AutoGPT. November 2024

    work page 2024

  2. [2]

    https://copilot.microsoft.com/

    Bing Copilot. https://copilot.microsoft.com/

    work page

  3. [3]

    https://blog.google/products/search/ generative-ai-google-search-may-2024/

    Generative ai in search: Let google do the searching for you. https://blog.google/products/search/ generative-ai-google-search-may-2024/

    work page 2024

  4. [4]

    https://www

    Google scales back AI search answers af- ter it told users to eat glue. https://www. washingtonpost.com/technology/2024/05/ 30/google-halt-ai-search/ . November 2024

    work page 2024

  5. [5]

    https://www.reddit.com/r/Pizza/comments/ 1a19s0/comment/c8t7bbp/?utm_source=share& utm_medium=web3x&utm_name=web3xcss&utm_ term=1&utm_content=share_button

    My cheese slides off the pizza too easily. https://www.reddit.com/r/Pizza/comments/ 1a19s0/comment/c8t7bbp/?utm_source=share& utm_medium=web3x&utm_name=web3xcss&utm_ term=1&utm_content=share_button. November 2024

    work page 2024

  6. [6]

    https://github.com/gkamradt/LLMTest_ NeedleInAHaystack

    Needle In A Haystack - Pressure Testing LLMs. https://github.com/gkamradt/LLMTest_ NeedleInAHaystack. November 2024

    work page 2024

  7. [7]

    https://openai.com/index/ new-and-improved-embedding-model/

    New and improved embedding model. https://openai.com/index/ new-and-improved-embedding-model/ . May 2025

    work page 2025

  8. [8]

    https://www.perplexity.ai/

    Perplexity AI. https://www.perplexity.ai/

    work page

Show all 75 references
  1. [9]

    https://www

    Why Google’s AI Overviews gets things wrong. https://www. technologyreview.com/2024/05/31/1093019/ why-are-googles-ai-overviews-results-so-bad/ #:~:text=In%20the%20case%20of%20AI,the% 20retrieval%20process%2C%20says%20Shah. November 2024

    2024

  2. [10]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv, 2023

    2023

  3. [11]

    Reliable, adaptable, and attributable language mod- els with retrieval

    Akari Asai, Zexuan Zhong, Danqi Chen, Pang Wei Koh, Luke Zettlemoyer, Hannaneh Hajishirzi, and Wen-tau Yih. Reliable, adaptable, and attributable language mod- els with retrieval. arXiv, 2024

    2024

  4. [12]

    Longbench: A bilingual, multi- task benchmark for long context understanding

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. Longbench: A bilingual, multi- task benchmark for long context understanding. arXiv preprint arXiv:2308.14508, 2023

    2023 arXiv

  5. [13]

    Evalu- ating the susceptibility of pre-trained language models via handcrafted adversarial examples

    Hezekiah J Branch, Jonathan Rodriguez Cefalu, Jeremy McHugh, Leyla Hujer, Aditya Bahl, Daniel del Castillo Iglesias, Ron Heichman, and Ramesh Darwishi. Evalu- ating the susceptibility of pre-trained language models via handcrafted adversarial examples. arXiv, 2022

    2022

  6. [14]

    Polyno- mial calculation of the shapley value based on sampling

    Javier Castro, Daniel Gómez, and Juan Tejada. Polyno- mial calculation of the shapley value based on sampling. Computers & operations research , 36(5):1726–1730, 2009

    2009

  7. [15]

    Xprompt: Explaining large language model’s generation via joint prompt attribution

    Yurui Chang, Bochuan Cao, Yujia Wang, Jinghui Chen, and Lu Lin. Xprompt: Explaining large language model’s generation via joint prompt attribution. arXiv preprint arXiv:2405.20404, 2024

    2024

  8. [16]

    Phan- tom: General trigger attacks on retrieval augmented lan- guage generation

    Harsh Chaudhari, Giorgio Severi, John Abascal, Matthew Jagielski, Christopher A Choquette-Choo, Mi- lad Nasr, Cristina Nita-Rotaru, and Alina Oprea. Phan- tom: General trigger attacks on retrieval augmented lan- guage generation. arXiv, 2024

    2024

  9. [17]

    Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases

    Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases. arXiv, 2024

    2024

  10. [18]

    Trojanrag: Retrieval-augmented generation can be back- door driver in large language models

    Pengzhou Cheng, Yidong Ding, Tianjie Ju, Zongru Wu, Wei Du, Ping Yi, Zhuosheng Zhang, and Gongshen Liu. Trojanrag: Retrieval-augmented generation can be back- door driver in large language models. arXiv, 2024

    2024

  11. [19]

    Beagle: Foren- sics of deep learning backdoor attack for better defense

    Siyuan Cheng, Guanhong Tao, Yingqi Liu, Shengwei An, Xiangzhe Xu, Shiwei Feng, Guangyu Shen, Kaiyuan Zhang, Qiuling Xu, Shiqing Ma, et al. Beagle: Foren- sics of deep learning backdoor attack for better defense. arXiv preprint arXiv:2301.06241, 2023

    2023 arXiv

  12. [20]

    Contextcite: At- tributing model generation to context

    Benjamin Cohen-Wang, Harshay Shah, Kristian Georgiev, and Aleksander Madry. Contextcite: At- tributing model generation to context. In NeurIPS, 2024

    2024

  13. [21]

    Characteriza- tions of an empirical influence function for detecting in- fluential cases in regression

    R Dennis Cook and Sanford Weisberg. Characteriza- tions of an empirical influence function for detecting in- fluential cases in regression. Technometrics, 22(4):495– 508, 1980

    1980

  14. [22]

    Explaining by removing: A unified framework for model explana- tion

    Ian Covert, Scott Lundberg, and Su-In Lee. Explaining by removing: A unified framework for model explana- tion. Journal of Machine Learning Research, 22(209):1– 90, 2021

    2021

  15. [23]

    Synthetic disinformation attacks on automated fact verification systems

    Yibing Du, Antoine Bosselut, and Christopher D Man- ning. Synthetic disinformation attacks on automated fact verification systems. In AAAI, 2022

    2022

  16. [24]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, 15 Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv, 2024

    2024

  17. [25]

    Textgenshap: Scalable post-hoc explanations in text generation with long documents

    James Enouen, Hootan Nakhost, Sayna Ebrahimi, Ser- can O Arik, Yan Liu, and Tomas Pfister. Textgenshap: Scalable post-hoc explanations in text generation with long documents. arXiv, 2023

    2023

  18. [26]

    Prompt Injection Attacks: A New Frontier in Cybersecurity

    Jacob Fox. Prompt Injection Attacks: A New Frontier in Cybersecurity. https://www.cobalt.io/blog/ prompt-injection-attacks, 2023

    2023

  19. [27]

    Enabling large language models to generate text with citations

    Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. Enabling large language models to generate text with citations. In EMNLP, 2023

    2023

  20. [28]

    Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injec- tion

    Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injec- tion. In AISec, 2023

    2023

  21. [29]

    Long-context llms meet rag: Overcoming chal- lenges for long inputs in rag

    Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan O Arik. Long-context llms meet rag: Overcoming chal- lenges for long inputs in rag. arXiv, 2024

    2024

  22. [30]

    Dense passage retrieval for open-domain question answering

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In EMNLP, 2020

    2020

  23. [31]

    The narrativeqa reading comprehension challenge

    Tomáš Koˇcisk`y, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. Transactions of the Association for Compu- tational Linguistics, 6:317–328, 2018

    2018

  24. [32]

    Natural questions: a benchmark for question answering research

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. TACL, 2019

    2019

  25. [33]

    Long context rag performance of large language models

    Quinn Leng, Jacob Portes, Sam Havens, Matei Zaharia, and Michael Carbin. Long context rag performance of large language models. arXiv, 2024

    2024

  26. [34]

    Retrieval-augmented generation for knowledge- intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge- intensive nlp tasks. NeurIPS, 2020

    2020

  27. [35]

    Autodan: Generating stealthy jailbreak prompts on aligned large language models

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv, 2023

    2023

  28. [36]

    Formalizing and benchmark- ing prompt injection attacks and defenses

    Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. Formalizing and benchmark- ing prompt injection attacks and defenses. In USENIX Security Symposium, 2024

    2024

  29. [37]

    A unified approach to interpreting model predictions

    Scott Lundberg. A unified approach to interpreting model predictions. arXiv, 2017

    2017

  30. [38]

    Using captum to explain generative language models

    Vivek Miglani, Aobo Yang, Aram Markosyan, Diego Garcia-Olano, and Narine Kokhlikyan. Using captum to explain generative language models. In NLP-OSS, 2023

    2023

  31. [39]

    Factscore: Fine- grained atomic evaluation of factual precision in long form text generation

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettle- moyer, and Hannaneh Hajishirzi. Factscore: Fine- grained atomic evaluation of factual precision in long form text generation. In EMNLP, 2023

    2023

  32. [40]

    Game theory

    Roger B Myerson. Game theory. Harvard university press, 2013

    2013

  33. [41]

    Yohei’s blog post

    Yohei Nakajima. Yohei’s blog post. https: //twitter.com/yoheinakajima/status/ 1582844144640471040, 2022

    2022

  34. [42]

    We- bgpt: Browser-assisted question-answering with human feedback

    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shan- tanu Jain, Vineet Kosaraju, William Saunders, et al. We- bgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021

    2021 arXiv

  35. [43]

    Ms marco: A human generated machine reading compre- hension dataset

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. Ms marco: A human generated machine reading compre- hension dataset. choice, 2640:660, 2016

    2016

  36. [44]

    On the risk of misinformation pollution with large language models

    Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Yang Wang. On the risk of misinformation pollution with large language models. In EMNLP, 2023

    2023

  37. [45]

    Neural exec: Learning (and learning from) execu- tion triggers for prompt injection attacks

    Dario Pasquini, Martin Strohmeier, and Carmela Tron- coso. Neural exec: Learning (and learning from) execu- tion triggers for prompt injection attacks. arXiv, 2024

    2024

  38. [46]

    Ignore previous prompt: Attack techniques for language models

    Fábio Perez and Ian Ribeiro. Ignore previous prompt: Attack techniques for language models. arXiv, 2022

    2022

  39. [47]

    Rise: Ran- domized input sampling for explanation of black-box models

    Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Ran- domized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421, 2018

    2018 arXiv

  40. [48]

    Estimating training data influence by tracing gradient descent

    Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. NeurIPS, 2020. 16

    2020

  41. [49]

    why should i trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explaining the predictions of any classifier. In KDD, 2016

    2016

  42. [50]

    Machine against the rag: Jamming retrieval-augmented generation with blocker documents

    Avital Shafran, Roei Schuster, and Vitaly Shmatikov. Machine against the rag: Jamming retrieval-augmented generation with blocker documents. arXiv, 2024

    2024

  43. [51]

    Poison forensics: Traceback of data poi- soning attacks in neural networks

    Shawn Shan, Arjun Nitin Bhagoji, Haitao Zheng, and Ben Y Zhao. Poison forensics: Traceback of data poi- soning attacks in neural networks. In USENIX Security, 2022

    2022

  44. [52]

    Learning important features through propagating activation differences

    Avanti Shrikumar, Peyton Greenside, and Anshul Kun- daje. Learning important features through propagating activation differences. In ICML, 2017

    2017

  45. [53]

    Deep inside convolutional networks: Visualising image classification models and saliency maps

    Karen Simonyan. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013

    2013 arXiv

  46. [54]

    Ax- iomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Ax- iomatic attribution for deep networks. In ICML, 2017

    2017

  47. [55]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  48. [56]

    Interactive prompt debug- ging with sequence salience

    Ian Tenney, Ryan Mullins, Bin Du, Shree Pandya, Min- suk Kahng, and Lucas Dixon. Interactive prompt debug- ging with sequence salience. arXiv, 2024

    2024

  49. [57]

    Regression shrinkage and selection via the lasso

    Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1):267–288, 1996

    1996

  50. [58]

    Musique: Multihop questions via single-hop question composition

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539– 554, 2022

    2022

  51. [59]

    Gradient based feature attribution in explainable ai: A technical review

    Yongjie Wang, Tong Zhang, Xu Guo, and Zhiqi Shen. Gradient based feature attribution in explainable ai: A technical review. arXiv, 2024

    2024

  52. [60]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS, 2022

    2022

  53. [61]

    Long-form factuality in large language models

    Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, et al. Long-form factuality in large language models. arXiv, 2024

    2024

  54. [62]

    Delimiters won’t save you from prompt injection

    Simon Willison. Delimiters won’t save you from prompt injection. https://simonwillison.net/2023/May/ 11/delimiters-wont-save-you . 2023

    2023

  55. [63]

    Prompt injection attacks against gpt-3

    Simon Willison. Prompt injection attacks against gpt-3. https://simonwillison.net/2022/Sep/12/ prompt-injection/. 2022

    2022

  56. [64]

    Prompt injection attacks against GPT-3

    Simon Willison. Prompt injection attacks against GPT-3. https://simonwillison.net/2022/Sep/ 12/prompt-injection/, 2022

    2022

  57. [65]

    Certifiably robust rag against retrieval corruption

    Chong Xiang, Tong Wu, Zexuan Zhong, David Wagner, Danqi Chen, and Prateek Mittal. Certifiably robust rag against retrieval corruption. arXiv, 2024

    2024

  58. [66]

    Bad- chain: Backdoor chain-of-thought prompting for large language models

    Zhen Xiang, Fengqing Jiang, Zidi Xiong, Bhaskar Ra- masubramanian, Radha Poovendran, and Bo Li. Bad- chain: Backdoor chain-of-thought prompting for large language models. arXiv, 2024

    2024

  59. [67]

    Badrag: Identifying vulnerabili- ties in retrieval augmented generation of large language models

    Jiaqi Xue, Mengxin Zheng, Yebowen Hu, Fei Liu, Xun Chen, and Qian Lou. Badrag: Identifying vulnerabili- ties in retrieval augmented generation of large language models. arXiv, 2024

    2024

  60. [68]

    Hotpotqa: A dataset for diverse, ex- plainable multi-hop question answering

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D Manning. Hotpotqa: A dataset for diverse, ex- plainable multi-hop question answering. In EMNLP, 2018

    2018

  61. [69]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In ICLR, 2023

    2023

  62. [70]

    Explainability for large language models: A survey

    Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1–38, 2024

    2024

  63. [71]

    Qmsum: A new benchmark for query-based multi-domain meeting summarization

    Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, et al. Qmsum: A new benchmark for query-based multi-domain meeting summarization. arXiv, 2021

    2021

  64. [72]

    Poisoning retrieval corpora by injecting adversarial passages

    Zexuan Zhong, Ziqing Huang, Alexander Wettig, and Danqi Chen. Poisoning retrieval corpora by injecting adversarial passages. arXiv, 2023

    2023

  65. [73]

    Universal and trans- ferable adversarial attacks on aligned language models

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and trans- ferable adversarial attacks on aligned language models. arXiv, 2023. 17

    2023

  66. [74]

    Poisonedrag: Knowledge corruption attacks to retrieval-augmented generation of large language mod- els

    Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia. Poisonedrag: Knowledge corruption attacks to retrieval-augmented generation of large language mod- els. In USENIX Security, 2025

    2025

  67. [75]

    >" means “more important than

    Guido Zuccon, Bevan Koopman, and Razia Shaik. Chat- gpt hallucinates when attributing answers. In Proceed- ings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, pages 46–51, 2023. A Additional Exp...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.