REVIEW 11 cited by
Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreaking
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this paper, we show that with the ability to jailbreak a GenAI model, attackers can escalate the outcome of attacks against RAG-based GenAI-powered applications in severity and scale. In the first part of the paper, we show that attackers can escalate RAG membership inference attacks and RAG entity extraction attacks to RAG documents extraction attacks, forcing a more severe outcome compared to existing attacks. We evaluate the results obtained from three extraction methods, the influence of the type and the size of five embeddings algorithms employed, the size of the provided context, and the GenAI engine. We show that attackers can extract 80%-99.8% of the data stored in the database used by the RAG of a Q&A chatbot. In the second part of the paper, we show that attackers can escalate the scale of RAG data poisoning attacks from compromising a single GenAI-powered application to compromising the entire GenAI ecosystem, forcing a greater scale of damage. This is done by crafting an adversarial self-replicating prompt that triggers a chain reaction of a computer worm within the ecosystem and forces each affected application to perform a malicious activity and compromise the RAG of additional applications. We evaluate the performance of the worm in creating a chain of confidential data extraction about users within a GenAI ecosystem of GenAI-powered email assistants and analyze how the performance of the worm is affected by the size of the context, the adversarial self-replicating prompt used, the type and size of the embeddings algorithm employed, and the number of hops in the propagation. Finally, we review and analyze guardrails to protect RAG-based inference and discuss the tradeoffs.
Forward citations
Cited by 11 Pith papers
-
External Data Extraction Attacks against Retrieval-Augmented Large Language Models
SECRET, an adaptive RAG extraction attack that fuses LLM-optimized jailbreak prompts with cluster-guided queries, extracts large portions of private databases from commercial and open LLMs, including ~35% of a sampled...
-
A Case Study on the Impact of Anonymization Along the RAG Pipeline
Anonymization placement in RAG—at the dataset or at the generated answer—creates observable differences in privacy protection versus response utility.
-
ADAM: A Systematic Data Extraction Attack on Agent Memory via Adaptive Querying
ADAM extracts data from LLM agent memory with up to 100% attack success rate by estimating data distribution and selecting queries via entropy guidance.
-
Graphs Don't Stay Secret: Practical Subgraph Reconstruction Attacks on Defended Graph RAG
GRASP, a closed-box multi-turn prompt attack, reconstructs type-faithful one-hop subgraphs from defended Graph RAG systems, reaching up to 82.9 F1 where prior attacks fail.
-
Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerous
Malicious calendar invites and emails can poison Gemini's context, enabling data exfiltration, app control, and physical-world actions.
-
Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelity
A multi-agent semantic rewriting system for RAG cuts targeted privacy leakage from 144 to 1 instances on LLaMA-3-8B while raising BLEU-1 to 0.122 over SAGE's 0.117, with offline preprocessing.
-
Detecting RAG Extraction Attack via Dual-Path Runtime Integrity Game
CanaryRAG detects RAG extraction attacks in real time by embedding canary tokens and checking dual target-oracle paths for integrity violations, achieving lower chunk recovery rates with negligible overhead.
-
ALDEN: Boosting Private Data Extraction from Retrieval-Augmented Generation Systems via Active Learning and Distribution Estimation
ALDEN boosts private data extraction rates from RAG systems by combining active learning for query diversification with dynamic estimation of the underlying knowledge-base topic distribution.
-
Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation
A unified benchmark comparing RAG knowledge-extraction attacks and defenses, showing query diversity boosts extraction, embedding attacks fail to transfer, and graph indexing raises per-token leakage.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
Provably Secure Retrieval-Augmented Generation
SAG encrypts RAG knowledge bases and claims formal security, but its proofs are flawed and its benchmarks guarantee zero attack success by design.
Discussion (0). Sign in to comment.