Pith. sign in

REVIEW 24 cited by

Beyond Memorization: Violating Privacy Via Inference with Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.07298 v2 pith:NFN3OZ7A submitted 2023-10-11 cs.AI cs.LG

Robin Staab , Mark Vero , Mislav Balunović , Martin Vechev This is my paper
classification cs.AIcs.LG
keywords privacyllmspersonalcurrentinferenceattributesinfermodels
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Current privacy research on large language models (LLMs) primarily focuses on the issue of extracting memorized training data. At the same time, models' inference capabilities have increased drastically. This raises the key question of whether current LLMs could violate individuals' privacy by inferring personal attributes from text given at inference time. In this work, we present the first comprehensive study on the capabilities of pretrained LLMs to infer personal attributes from text. We construct a dataset consisting of real Reddit profiles, and show that current LLMs can infer a wide range of personal attributes (e.g., location, income, sex), achieving up to $85\%$ top-1 and $95\%$ top-3 accuracy at a fraction of the cost ($100\times$) and time ($240\times$) required by humans. As people increasingly interact with LLM-powered chatbots across all aspects of life, we also explore the emerging threat of privacy-invasive chatbots trying to extract personal information through seemingly benign questions. Finally, we show that common mitigations, i.e., text anonymization and model alignment, are currently ineffective at protecting user privacy against LLM inference. Our findings highlight that current LLMs can infer personal data at a previously unattainable scale. In the absence of working defenses, we advocate for a broader discussion around LLM privacy implications beyond memorization, striving for a wider privacy protection.

Discussion (0). Sign in to comment.

Forward citations

Cited by 24 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. OCELOT: Inference-Leakage Budgets for Privacy-Preserving LLM Agents

    cs.CR 2026-06 unverdicted novelty 7.0 of 10

    OCELOT recasts agent privacy as posterior-risk control and implements Witness-Verified Declassification to authorize the least-disclosing useful release under a sink-trust-weighted min-entropy budget.

  2. CachePrune: Privacy-Aware and Fine-Grained KV Cache Sharing for Efficient LLM Inference

    cs.CR 2026-05 unverdicted novelty 7.0 of 10

    CachePrune enables fine-grained, token-level KV cache reuse across LLM requests by masking sensitive segments, eliminating direct side-channel leakage while cutting TTFT by 4.5x and raising hit rates by 44% versus pri...

  3. When Are LLM Inferences Acceptable? User Reactions and Control Preferences for Inferred Personal Information

    cs.HC 2026-05 unverdicted novelty 7.0 of 10

    Users show curiosity over concern toward LLM inferences of personal information, with acceptability depending on context, alignment with expectations, and who uses the inferences rather than just the content.

  4. Tracking Conversations: Measuring Content and Identity Exposure on AI Chatbots

    cs.CR 2026-04 accept novelty 7.0 of 10

    17 of 20 AI chatbots share conversation content or identifiers with third parties, including plaintext prompt and response text with Microsoft Clarity in three cases.

  5. Tracking Conversations: Measuring Content and Identity Exposure on AI Chatbots

    cs.CR 2026-04 unverdicted novelty 7.0 of 10

    17 of 20 AI chatbots share conversation content or identifiers with third parties, including plaintext text sent to Microsoft Clarity via session replay in three cases.

  6. Understanding User Privacy Perceptions of GenAI Smartphones

    cs.CR 2026-04 unverdicted novelty 7.0 of 10

    Users show limited grasp of GenAI smartphone data practices but heightened privacy concerns across collection, storage, and control, calling for better transparency and user controls.

  7. Understanding the Supply Chain and Risks of Large Language Model Applications

    cs.SE 2025-07 conditional novelty 7.0 of 10

    A new benchmark dataset traces dependencies across 3,859 LLM applications, 109,211 models, 2,474 datasets, and 8,862 libraries, and finds widespread known vulnerabilities in application dependencies.

  8. Agentic AI-Powered Re-Identification: An Emerging, Scalable Threat to Mobility Microdata Privacy

    cs.CR 2026-06 conditional novelty 6.0 of 10

    Agentic AI re-identifies 72% of individuals from simulated mobility traces by cross-referencing public web sources without human intervention.

  9. Trustworthy Recommendation in the Era of Large Language Models: Opportunities and Challenges

    cs.IR 2026-05 unverdicted novelty 6.0 of 10

    A systematic review of over 200 studies concludes that LLMs in recommender systems act as a double-edged sword, creating both opportunities and new risks for trustworthiness.

  10. Inferential Privacy Leakage in Anonymized Conversational AI Logs

    cs.CY 2026-05 unverdicted novelty 6.0 of 10

    LLM-based inference recovers user age, gender, and country from filtered ChatGPT logs at weighted F1 scores of 0.84-0.90, with median identification from the first 5% of history, driven by stereotype patterns.

  11. Profiling for Pennies: Unveiling the Privacy Iceberg of LLM Agents

    cs.CR 2026-05 unverdicted novelty 6.0 of 10

    LLM agents can reconstruct high-fidelity personal profiles from minimal PII seeds with over 90% accuracy in under 10 minutes at less than $3 cost, exposing three escalating tiers of privacy risks.

  12. PAC-BENCH: Evaluating Multi-Agent Collaboration under Privacy Constraints

    cs.AI 2026-04 unverdicted novelty 6.0 of 10

    Privacy constraints degrade multi-agent AI collaboration performance through recurring coordination breakdowns such as early privacy violations, overly conservative abstraction, and privacy-induced hallucinations.

  13. MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models

    cs.CV 2025-11 conditional novelty 6.0 of 10

    A new nine-task bilingual benchmark shows most tested vision-language models can link fragmented images and documents into a specific individual's profile, with top models scoring above 0.85 on overall reasoning risk.

  14. Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning

    cs.LG 2025-10 conditional novelty 6.0 of 10

    Downgrading optimizers to lower-information variants during LLM unlearning yields more robust forgetting on MUSE and WMDP benchmarks by converging to harder-to-perturb loss basins.

  15. User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies

    cs.CY 2025-09 conditional novelty 6.0 of 10

    All six leading U.S. AI chatbot developers, as of May 2025, appear to train their models on users' chat data by default, often without clear opt-out options.

  16. What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests

    cs.CL 2025-07 conditional novelty 6.0 of 10

    WikiMem, a Wikidata-derived canary dataset and a calibrated NLL-ranking metric, identifies which human-fact associations an LLM has memorized, with higher rates for famous people and larger models.

  17. Imperceptible and Reversible Adversarial Examples against Vision-Language Models for Privacy Protection

    cs.CV 2026-07 conditional novelty 5.5 of 10

    CloakDiff generates high-fidelity reversible adversarial images that suppress VLM text-query privacy leakage via diffusion attention editing plus invertible steganography.

  18. PromptPrint: Behavioral Biometrics Through Natural Language Prompting in LLMs

    cs.CL 2026-06 unverdicted novelty 5.0 of 10

    Short LLM prompts contain distinctive lexical signals enabling user identification as a behavioral biometric, with lexical features outperforming semantic ones across a dataset of 20k+ prompts from 1k users.

  19. PromptArmor: Simple yet Effective Prompt Injection Defenses

    cs.CR 2025-07 conditional novelty 5.0 of 10

    PromptArmor prompts a separate LLM to detect and remove injected prompts from agent inputs, achieving below 1% false positive and false negative rates on AgentDojo.

  20. Selective Token-Level Cryptographic Redaction for Privacy-Preserving Clinical Deployment of Large Language Models

    cs.CL 2026-06 unverdicted novelty 4.0 of 10

    HERALD selectively encrypts sensitive tokens via medical NER, POS policies, and deterministic ciphertext substitution to enable privacy-preserving clinical LLM use while recovering near-plaintext task performance.

  21. Generative Data Refinement: Just Ask for Better Data

    cs.LG 2025-09 conditional novelty 4.0 of 10

    A pretrained LLM can rewrite individual data samples to strip out PII or toxic content while preserving useful information, creating safer training data.

  22. SLM Finetuning for Natural Language to Domain Specific Code Generation in Production

    cs.LG 2026-04 unverdicted novelty 3.0 of 10

    Fine-tuned small language models outperform larger models in natural language to domain-specific code generation with improved performance, latency, and the ability to adapt to customer-specific scenarios without losi...

  23. A Vision Toward Energy-Efficient Domain-Specific Artificial Intelligence Models and Agents

    cs.AI 2025-10 unverdicted novelty 2.0 of 10

    A position paper proposing compact, domain-specific AI agents as the path to ≥1000× energy efficiency, without demonstrating the claim.

  24. A Survey on Data Security in Large Language Models

    cs.CR 2025-08 conditional novelty 2.0 of 10

    A survey of data security risks in LLMs that organizes threats, defenses, and evaluation datasets, with notable factual errors in its tables.

Pith tools