REVIEW 8 cited by
Auditing Prompt Caching in Language Model APIs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Auditing Prompt Caching in Language Model APIs
read the original abstract
Prompt caching in large language models (LLMs) results in data-dependent timing variations: cached prompts are processed faster than non-cached prompts. These timing differences introduce the risk of side-channel timing attacks. For example, if the cache is shared across users, an attacker could identify cached prompts from fast API response times to learn information about other users' prompts. Because prompt caching may cause privacy leakage, transparency around the caching policies of API providers is important. To this end, we develop and conduct statistical audits to detect prompt caching in real-world LLM API providers. We detect global cache sharing across users in seven API providers, including OpenAI, resulting in potential privacy leakage about users' prompts. Timing variations due to prompt caching can also result in leakage of information about model architecture. Namely, we find evidence that OpenAI's embedding model is a decoder-only Transformer, which was previously not publicly known.
Forward citations
Cited by 8 Pith papers
-
Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing
Per-token generation timing leaks speculative decoding and draft-model context length from Gemini, and recovers layer count and hidden size of Llama-family models with top-5 accuracy up to 65% when both are unknown.
-
Cache-Aware Prompt Compression:A Two-Tier Cost Model for LLM API Caching
CAPC combines query-agnostic compression with prompt caching and is reported cheapest in 16/16 LongBench configurations, but the empirical support has unresolved specification and consistency gaps.
-
Behavioral Consistency and Transparency Analysis on Large Language Model API Gateways
GateScope auditing of 10 LLM gateways finds frequent issues including silent model substitutions, degraded memory retention, pricing deviations, and latency instability.
-
PrefixWall: Mitigating Prefix Caching Side Channels in Shared LLM Systems
PrefixWall mitigates APC side channels in multi-tenant LLM systems via selective prefix isolation, delivering up to 70% higher cache reuse and 30% lower latency than full-isolation baselines.
-
Security Considerations for Multi-agent Systems
No existing AI security framework covers a majority of the 193 identified multi-agent system threats in any category, with OWASP Agentic Security Initiative achieving the highest overall coverage at 65.3%.
-
From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching
An attacker can force false cache-key collisions in LLM semantic caching and hijack responses or agent tool calls, with 86% hit rate in experiments.
-
A Study of LLMs' Preferences for Libraries and Programming Languages
Empirical study of eight LLMs finds overuse of popular libraries like NumPy in up to 45% of unnecessary cases and strong default preference for Python even when suboptimal.
-
The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes
A literature survey that introduces a taxonomy for LLM reasoning paradigms, analyzes methodological trends, and synthesizes failure modes from over 300 papers.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.