REVIEW 6 cited by
Prompt Leakage effect and defense strategies for multi-turn LLM interactions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Prompt leakage poses a compelling security and privacy threat in LLM applications. Leakage of system prompts may compromise intellectual property, and act as adversarial reconnaissance for an attacker. A systematic evaluation of prompt leakage threats and mitigation strategies is lacking, especially for multi-turn LLM interactions. In this paper, we systematically investigate LLM vulnerabilities against prompt leakage for 10 closed- and open-source LLMs, across four domains. We design a unique threat model which leverages the LLM sycophancy effect and elevates the average attack success rate (ASR) from 17.7% to 86.2% in a multi-turn setting. Our standardized setup further allows dissecting leakage of specific prompt contents such as task instructions and knowledge documents. We measure the mitigation effect of 7 black-box defense strategies, along with finetuning an open-source model to defend against leakage attempts. We present different combination of defenses against our threat model, including a cost analysis. Our study highlights key takeaways for building secure LLM applications and provides directions for research in multi-turn LLM interactions
Forward citations
Cited by 6 Pith papers
-
Mitigating Over-Personalization in LLMs via Structured Memory
Grouping a user's stored memories into domain-labeled sections before injecting them into an LLM's context reduces cross-domain leakage by 8.8 percentage points on average across seven models, while slightly improving...
-
In-Context Learning for Wound Classification with Small Multimodal Language Models
Retrieval-based in-context learning, not zero-shot prompting, drives wound-classification gains in small multimodal models, with Qwen 3.5 27B reaching 0.872 accuracy on Kaggle and 0.678 on Medetec.
-
LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems
A systematic review that categorizes LLM threats, severity scores, and mitigations across development and operation life cycles and multiple deployment scenarios.
-
Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerous
Malicious calendar invites and emails can poison Gemini's context, enabling data exfiltration, app control, and physical-world actions.
-
PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization
A gradient-based iterative in-context optimization framework, PIG, extracts personally identifiable information from aligned LLMs with high success rates on white-box and black-box models.
-
Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents
A survey proposing a source-and-impact taxonomy (input, model, combined; security, privacy, ethics) for threats to LLM-based agents, with feature analysis and four case studies.
Discussion (0). Continue with ORCID to comment.