REVIEW 31 cited by
BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models
read the original abstract
Large Language Models (LLMs) are constrained by outdated information and a tendency to generate incorrect data, commonly referred to as "hallucinations." Retrieval-Augmented Generation (RAG) addresses these limitations by combining the strengths of retrieval-based methods and generative models. This approach involves retrieving relevant information from a large, up-to-date dataset and using it to enhance the generation process, leading to more accurate and contextually appropriate responses. Despite its benefits, RAG introduces a new attack surface for LLMs, particularly because RAG databases are often sourced from public data, such as the web. In this paper, we propose \TrojRAG{} to identify the vulnerabilities and attacks on retrieval parts (RAG database) and their indirect attacks on generative parts (LLMs). Specifically, we identify that poisoning several customized content passages could achieve a retrieval backdoor, where the retrieval works well for clean queries but always returns customized poisoned adversarial queries. Triggers and poisoned passages can be highly customized to implement various attacks. For example, a trigger could be a semantic group like "The Republican Party, Donald Trump, etc." Adversarial passages can be tailored to different contents, not only linked to the triggers but also used to indirectly attack generative LLMs without modifying them. These attacks can include denial-of-service attacks on RAG and semantic steering attacks on LLM generations conditioned by the triggers. Our experiments demonstrate that by just poisoning 10 adversarial passages can induce 98.2\% success rate to retrieve the adversarial passages. Then, these passages can increase the reject ratio of RAG-based GPT-4 from 0.01\% to 74.6\% or increase the rate of negative responses from 0.22\% to 72\% for targeted queries.
Forward citations
Cited by 31 Pith papers
-
DiscourseFlip: An Oblique Discourse-Level Opinion Manipulation Attack against Black-box Retrieval-Augmented Generation
DiscourseFlip is a graph-guided attack allocating limited poisoning budget to induce targeted opinion shifts over semantic query networks in black-box RAG.
-
Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoning
Oracle Poisoning corrupts knowledge graphs used by AI agents via tool calls, leading tested models to accept fabricated claims at 100% under directed queries in a production-scale demonstration.
-
ShadowMerge: A Novel Poisoning Attack on Graph-Based Agent Memory via Relation-Channel Conflicts
ShadowMerge poisons graph-based agent memory via relation-channel conflicts using an AIR pipeline, achieving 93.8% average attack success rate on Mem0 and three real-world datasets while bypassing existing defenses.
-
ShadowMerge: A Novel Poisoning Attack on Graph-Based Agent Memory via Relation-Channel Conflicts
ShadowMerge exploits relation-channel conflicts to poison graph-based agent memory, achieving 93.8% average attack success rate on Mem0 and real-world datasets while bypassing existing defenses.
-
ShadowMerge: A Novel Poisoning Attack on Graph-Based Agent Memory via Relation-Channel Conflicts
ShadowMerge poisons graph-based agent memory by creating relation-channel conflicts that get extracted and retrieved, achieving 93.8% attack success rate on Mem0 and datasets like PubMedQA while evading prior defenses.
-
Salience Induction against Multi-Hop RAG Agents: Threat and Defense
A new class of attack on retrieval-augmented AI: reordering and rephrasing true facts in documents flips an agent's reasoning steps, reaching 83.3% attack success with no false claims injected.
-
REALM: A Unified Red-Teaming Benchmark for Physical-World VLMs
REALM is the first unified red-teaming benchmark for physical-world VLMs that aligns diverse attack methods via an agentic target-generation pipeline and evaluates them on shared datasets showing text/typographic atta...
-
Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
LLM recommendation systems show strong bias toward well-known brands that can be overcome by small rating advantages or authority marketing claims, creating a social dilemma when multiple brands optimize.
-
ImageAuditor: Membership Inference Attack against Image-based Retrieval-Augmented Generation
ImageAuditor is the first MIA for IRAG that achieves over 80% AUROC with four queries by using reward-guided policy optimization for cross-modal retrieval and task-specific prompting for signal extraction.
-
SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoning
SilentRetrieval is a data poisoning attack achieving 84.6% HR@10 and 57.5% ASR-LLM on Natural Questions via coordinated beam search and trigger fusion while preserving document fluency.
-
BadSKP: Backdoor Attacks on Knowledge Graph-Enhanced LLMs with Soft Prompts
BadSKP poisons graph node embeddings to steer soft prompts in KG-enhanced LLMs, achieving high attack success rates where text-channel backdoors fail due to semantic anchoring.
-
MEMSAD: Gradient-Coupled Anomaly Detection for Memory Poisoning in Retrieval-Augmented Agents
MEMSAD links anomaly detection gradients to retrieval objectives under encoder regularity to certify detection of continuous memory poisons, achieving perfect TPR/FPR in experiments while exposing a synonym-invariance gap.
-
MEMSAD: Gradient-Coupled Anomaly Detection for Memory Poisoning in Retrieval-Augmented Agents
MEMSAD uses a provable gradient coupling between anomaly detection and retrieval objectives to deliver certified detection of memory poisoning in LLM agents, achieving optimal sample complexity and perfect TPR/FPR in ...
-
Needle-in-RAG: Prompt-Conditioned Character-Level Traceback of Poisoned Spans in Retrieved Evidence
RAGCharacter localizes poisoned character spans in RAG evidence via prompt-conditioned counterfactual masking and achieves the best accuracy-over-attribution trade-off across tested attacks and models.
-
Catching the Infection Before It Spreads: Foresight-Guided Defense in Multi-Agent Systems
A foresight-based local purification method using multi-persona simulations and recursive diagnosis reduces infectious jailbreak spread in multi-agent systems from over 95% to below 5.47% while matching benign perform...
-
Adversarial Hubness in Multi-Modal Retrieval
Adversarial hubs can be generated to be retrieved as top-1 for over 84% of test queries in text-to-image retrieval, far exceeding natural hubs.
-
Trustworthiness in Retrieval-Augmented Generation Systems: A Survey
Introduces Trust-RAG Compass framework and TRC Bench benchmark to assess RAG trustworthiness across factuality, robustness, fairness, transparency, accountability, and privacy, with evaluations showing performance gap...
-
A Failure-Mode Benchmark for Polymorphic Sybil Poisoning in RAG
Polymorphic sybil groups of six diverse passages amplify hijack rates 5.7× over monomorphic copies under Forced Exposure and leave 47–66% of outputs in unmonitored abstention or drift.
-
Conflict-Aware Retriever Editing for Knowledge Injection Attacks on LLM-Based RAG Systems
CAREATTACK adapts closed-form parameter editing with graph-based conflict resolution and lightweight anchor repair to promote malicious passages in RAG retrieval while limiting side effects on non-target queries.
-
RADAR: Defending RAG Dynamically against Retrieval Corruption
RADAR defends RAG systems in dynamic settings by framing reliable context selection as a Max-Flow Min-Cut graph problem with Bayesian memory updates, claiming superior robustness, response quality, and low storage on ...
-
Catching the Infection Before It Spreads: Foresight-Guided Defense in Multi-Agent Systems
FLP uses multi-persona foresight simulation to detect infections via response diversity and applies local purification to reduce maximum cumulative infection rates in multi-agent systems from over 95% to below 5.47%.
-
Catching the Infection Before It Spreads: Foresight-Guided Defense in Multi-Agent Systems
A foresight-based local purification method simulates future agent interactions, detects infections via response diversity across personas, and applies targeted rollback or recursive diagnosis to cut maximum infection...
-
Epistemic Bias Injection: Manipulating LLM Opinion via Selective Context Retrieval
Factually correct but one-sided passages injected into a RAG database can shift LLM answers toward an attacker's stance, and a PCA/KL-based filter (BiasDef) reduces that shift.
-
ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation
Documentation poisoning with hidden ranking and suggestion sequences can make RAG-based code generators confidently recommend malicious dependencies, even at 0.01% poisoning ratios.
-
Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation
Cross-lingual RACG shows non-trivial but unequal knowledge transfer across 13 programming languages, depending on linguistic affinity and pretraining diversity, with limited reliance on natural language information wh...
-
Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability
A role-layer survey unifies LLM misuse, LLM-based defense, and LLM-centric verification vulnerabilities across content, social, evidence, and workflow layers, then lists three open challenges.
-
Evaluating RAG Reliability under Clean, Misleading, and Mixed Retrieval
Proposes an evaluation framework using parametric override and confidence metrics to assess RAG robustness to clean, poisoned, and mixed retrieval evidence on factoid questions.
-
PRA-RAG: Provably Robust Aggregation in Retrieval-Augmented Generation against Retrieval Corruption
PRA-RAG is a new aggregation algorithm for RAG that claims provable robustness bounds against poisoned retrieved texts and reduces attack success rate to 1% while keeping 71% accuracy.
-
Agents at Risk: How Users Unwittingly Undermine LLM Safety
Commercial AI agents routinely trust user-relayed unverified content and execute risky actions unless the user explicitly demands a safety check.
-
Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems
A survey of architectures, threats, defenses, and future directions for security and privacy in RAG systems.
-
RAG Security and Privacy: Formalizing the Threat Model and Attack Surface
A formal RAG threat model is defined with four adversary classes and game-based notions of membership inference, leakage, and poisoning, but the definitions largely restate known concepts and the main DP-based protect...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.