Pith. sign in

REVIEW 3 major objections 4 minor 35 references

PANOPTICON offers the first benchmark for studying prompt-inversion attacks on private LLM context data.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 21:12 UTC pith:YT7ANEQN

load-bearing objection A genuinely useful synthetic PII-prompt corpus for inference-time privacy research, but the regex-based ground-truth labels are admitted-weak and the dataset isn't actually released, so the 'benchmark' claim is premature. the 3 major comments →

arxiv 2607.22695 v1 pith:YT7ANEQN submitted 2026-07-17 cs.AI cs.CR

PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window

Ryan Thornton , Mir Mehedi Ahsan Pritom , Maanak Gupta This is my paper
classification cs.AI cs.CR
keywords PII leakageprompt inversion attackssynthetic datasetLLM privacyinference-time privacybenchmark datasetcontext windowPII-F1 metric
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces PANOPTICON, a synthetic dataset of 67,718 prompts that contain labeled personally identifiable information (PII), built to study how LLMs leak private data at inference time rather than through memorization of training data. The authors argue that while real PII cannot be ethically released, a synthetic but naturalistic corpus can serve as a stand-in. They characterize the dataset's lexical and semantic diversity, and run a case study with a lightweight prompt-inversion attack to show how the data enables measuring PII-specific recovery. If the benchmark holds up, it would give privacy researchers a repeatable way to compare PII leakage across models, prompt types, and attack settings.

Core claim

The central claim is that PANOPTICON is the first benchmark dataset specifically designed for studying prompt-inversion attacks over private corpora. The dataset contains 67,718 prompts, each labeled with PII spans across six categories plus a benign control, derived from 9,674 synthetic user profiles. The paper measures lexical and semantic diversity to support the claim of naturalistic variety, and demonstrates a case study in which a white-box inversion probe recovers PII with a distinct PII-F1 score separate from token accuracy. The intended contribution is a foundation: a labeled, controllable corpus that future privacy-leakage research can use to quantify how much PII can be extracted

What carries the argument

The generation pipeline is the load-bearing mechanism: it takes synthetic user profiles, selects a PII category and scenario, constructs a system prompt that instructs an LLM to produce a first-person request naturally weaving in specified PII, and then labels the resulting prompt with a regular-expression tagger. The dataset's value rests on this pipeline producing diverse, naturalistic prompts with known ground-truth PII spans. The case study adds a second piece of machinery: a PII-F1 metric that scores recovery of PII spans separately from general token-level accuracy, so privacy leakage is measured by entity fidelity rather than surface text overlap.

Load-bearing premise

The ground-truth PII labels are produced by a regular-expression tagger whose accuracy is load-bearing; if the tagger mislabels spans, every statistic derived from the dataset—including the case-study PII-F1 numbers—inherits that error.

What would settle it

Take a random sample of, say, 500 PANOPTICON prompts, have two human annotators mark the PII spans, and measure agreement with the regex tagger. If agreement is low (e.g., F1 below 0.8), the benchmark's ground truth is unreliable and any conclusion drawn from it—including the claim of being a usable benchmark—needs revision.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Privacy researchers get an ethically usable, labeled corpus for inference-time leakage studies without exposing real individuals.
  • The dataset's category and scenario structure supports controlled comparisons of PII recoverability across prompt types (e.g., finance vs. health) and across models.
  • PII-F1 provides a leakage metric that is not conflated with paraphrase or boilerplate similarity, addressing a gap in prior prompt-inversion evaluations.
  • The pipeline can be re-parameterized for different PII distributions or model targets, so future benchmarks can be tailored to specific domains.
  • The case study's low absolute recovery numbers provide a lower-bound reference point for stronger attackers to beat.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the regex tagger's ground-truth labels are noisy, the reported PII presence rates and PII-F1 scores are biased; a human-annotated or model-annotated subset would test this directly.
  • The dataset could be reused beyond inversion attacks—for PII detection/redaction training, or for measuring contextual-integrity violations—since it contains labeled spans in realistic prompts.
  • The low PII-F1 relative to token-accuracy in the case study may suggest that PII spans (e.g., emails, phone numbers) are harder to invert than ordinary tokens, a hypothesis the dataset is designed to test across categories.
  • The paper's 'benchmark' claim currently rests on internal diversity metrics; external validation against other corpora would clarify whether PANOPTICON's distributions match real-world prompt PII usage.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces PANOPTICON, a dataset of 67,718 synthetic prompts generated with Llama-3.1-8B-Instruct from 9,674 PANORAMA synthetic user profiles, with span-level PII labels produced by a regular-expression tagger. The authors characterize the dataset with lexical and S-BERT diversity metrics, report PII presence distributions, and run a small case study adapting a white-box prompt-inversion attack to compute a PII-F1 recovery metric. The central claim is that PANOPTICON is the first benchmark dataset for studying prompt inversion attacks over private corpora, providing a controlled, labeled corpus for inference-time privacy research.

Significance. If fully supported, PANOPTICON would fill a real gap: there are few public, ethically constructed PII-laden prompt corpora for studying non-parametric privacy leakage, and the use of PANORAMA's synthetic profiles provides an independent, credited foundation. The pipeline is straightforward and mostly transparent, and the authors are explicit about many limitations. The PII-F1 metric is a sensible addition for leakage-focused evaluation. However, the central claim is currently contingent on the reliability of the regex-based ground-truth labels, which the paper itself admits are error-prone; the realism/diversity claims lack any comparison baseline; and the case study has very small samples and no error bars. These issues are load-bearing for the dataset's stated purpose, so the contribution is not yet at the level claimed.

major comments (3)
  1. [§III-C, §V-A, Table I, Table II] The ground-truth PII labels are produced by a regex tagger whose accuracy is load-bearing. The paper explicitly states in §III-C that 'the deterministic nature of our PII tagging mechanism may have overly captured PII in some instances, while under capturing PII in others,' and in §V-A that this 'diminishes ground-truth accuracy.' Every downstream quantity — the PII-presence rates in Table I and the PII-F1 recovery scores in Table II — is computed against these labels. If the tagger marks non-PII spans as PII (the address-number example in §III-C) or misses paraphrased PII, the central claim of a controlled ground-truth benchmark is unsupported. The paper should either validate label accuracy against the known source PII pool via exact/normalized string matching (which is feasible, since the PII was inserted from PANORAMA profiles) or provide an annotation study and per-label error analy
  2. [§III-C (diversity metrics)] The realism/diversity claims are not calibrated. The TTR of 0.016 is said to be 'consistent with large, naturalistic text corpora,' but no comparison corpus, reference, or statistical test is provided. The average pairwise S-BERT cosine distance of 0.83 is asserted to indicate semantic well-separation, but there is no baseline (e.g., real user prompts, other synthetic corpora, or a random/template baseline) against which this value can be interpreted. The near-duplicate threshold of 0.05 is also arbitrary. For a benchmark whose abstract claims 'realism,' these metrics need external grounding; otherwise they only show internal properties of the generated text, not realism.
  3. [§IV-B/C, Table II] The case study uses only 50–500 prompts and reports no variance, confidence intervals, or repeated-seed runs. The two PII-F1 values (0.0242 and 0.0044) differ by a factor of five between refinement settings, but without error bars or statistical testing it is impossible to tell whether this is meaningful or noise. The claim that the setup provides a 'consistent, repeatable leakage measurement framework' is therefore not demonstrated. Additionally, the comparison row for Qu et al. uses a different model (LLaMA-65B base) and a different dataset (Skytrax), so the side-by-side in Table II is not an apples-to-apples comparison; the authors acknowledge this, but the table's presentation is still easy to over-read.
minor comments (4)
  1. [Throughout] There are several typos and inconsistent spellings: 'Unfortunatley' (Section I), 'PANAROMA' vs 'PANORAMA' (Section III-A and elsewhere), and 'Specifcally' (Section IV). Please correct these.
  2. [Figure 2] The PCA projection figure has no legend or axis-label explanation, and the 'four distinct groupings' are not connected to any column in the dataset (e.g., category or scenario). Please clarify what the clusters correspond to and how the reader should interpret them.
  3. [Conclusion/Future Work] The paper states 'We release PANOPTICON and the accompanying generation and evaluation pipeline,' but no URL or repository is provided. For a benchmark paper, a release link or a clear statement of availability (e.g., anonymous for review) is needed.
  4. [§III-C] The claim that 'all PII categories have, at least, a 93.44% PII presence' is followed by Table I where the Relationship and Government rows both show 93.44%, while the Health row shows 95.64%. The table is consistent, but the sentence is slightly awkward because the minimum is shared by two rows. Consider rewording.

Circularity Check

2 steps flagged

Ground-truth labels are validated by the same regex tagger that produced them; realism is asserted from diversity metrics computed on the corpus itself.

specific steps
  1. self definitional [Section III-C, ground-truth labeling paragraph; also Section V-A]
    "After the prompt is generated, a function checks the prompts for PII utilizing our defined rules. ... the deterministic nature of our PII tagging mechanism may have overly captured PII in some instances, while under capturing PII in others. ... all PII categories have, at least, a 93.44% PII presence while the benign category only contains 8.45% prompts with PII. This indicates broad compliance with PII constraints, and thus still provides meaningful ground-truth for future experiments."

    The meaningfulness of the regex-based ground-truth labels is asserted from the very same tagger's aggregate counts (93.44% / 8.45%). The tagger's output is the source of both the labels and the compliance statistics used to validate those labels. Since the paper admits the tagger over- and under-captures PII, the compliance rate is not independent evidence; it is the tagger reporting on itself. The downstream PII-F1 case study is then scored against these unvalidated labels, so PII recoverability results inherit the tagger's unmeasured error modes.

  2. other [Section III-C, dataset composition and diversity paragraphs]
    "We measure the lexical diversity of the dataset using token-level statistics computed over the entire dataset. ... Together, these lexical and semantic metrics indicate that PANOPTICON exhibits high diversity at both the surface and meaning levels. ... these results support its use as a realistic and robust benchmark for studying PII leakage under prompt inversion attacks."

    The 'realism' claim is supported only by diversity statistics (TTR, entropy, S-BERT cosine distance) computed on PANOPTICON itself, with no external corpus, human evaluation, or independent benchmark. Diversity of a generated corpus is a descriptive property; by itself it does not establish that prompts resemble real user prompts. The validation is therefore self-referential: the corpus is judged realistic because it has high internal dispersion, a property that the generation procedure (random scenario sampling over 9,674 profiles) is designed to produce.

full rationale

The paper does not rely on self-citation chains: the source profiles come from the independent PANORAMA dataset and the PIA framework is adapted from Qu et al. The dataset generation itself is not circular, because PII values are drawn from known PANORAMA profiles. However, the paper's internal validation of its central artifact is partially self-referential. The ground-truth labels are produced by an in-house regex tagger whose error modes are admitted, yet the tagger's own aggregate statistics are used to conclude that the labels are 'meaningful ground-truth.' Similarly, the 'realistic and robust benchmark' conclusion is based on diversity metrics computed on the corpus itself, without comparison to any external reference. These are validation circularities rather than mathematical derivations, so they do not void the dataset's independent content, but they weaken the benchmark claims and make the case-study PII-F1 interpretable only relative to the tagger's operational definition of PII.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The central claims rest on several hand-chosen design decisions (source profiles, generation templates, regex labels, diversity thresholds) that are not derived or externally validated. The paper's own limitations section admits label inaccuracy and lack of real-world data.

free parameters (4)
  • near-duplicate cosine threshold = 0.05
    Prompts with pairwise S-BERT cosine distance below 0.05 are classified as near-duplicates; no sensitivity analysis or external basis is given.
  • refinement hyperparameters (K, m, R, Δ) = values not reported
    The PIA proxy's top-K candidate list, m candidates per position, 1–3 refinement rounds, and acceptance margin Δ are fixed by hand and affect the reported recovery rates.
  • regex PII rules = not enumerated
    The regular-expression rules determine all ground-truth PII labels; over- and under-capture is acknowledged in Section III-C.
  • generation prompt templates = hand-designed
    The system/user prompts in Appendix A and constraints in Appendix B shape the distribution of generated prompts; no evidence they match real user prompt distributions.
axioms (5)
  • domain assumption PANORAMA synthetic profiles are representative of real user PII distributions
    All 9,674 source profiles come from PANORAMA; the realism of PANOPTICON inherits PANORAMA's realism, which the paper does not verify.
  • domain assumption Llama-3.1-8B-Instruct writes naturalistic first-person prompts
    The entire dataset is machine-generated; no human evaluation is reported.
  • ad hoc to paper Regex matching suffices to identify PII spans
    The paper uses regex for labels and concedes that it over- or under-captures PII.
  • domain assumption Diversity metrics proxy for realism
    TTR, entropy, and S-BERT distances describe the generated corpus but are not compared to real prompt corpora or human judgments.
  • domain assumption Nearest-neighbor embedding inversion approximates real white-box PIA
    The case study uses a lightweight proxy and reports much lower accuracy than Qu et al.; whether it provides a 'lower bound' on PII recoverability is assumed, not proven.

pith-pipeline@v1.3.0-alltime-deepseek · 11718 in / 11967 out tokens · 93405 ms · 2026-08-01T21:12:01.602364+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window." pith.science (2026). https://pith.science/paper/YT7ANEQN

@misc{pith2026260722695,
  author       = {Pith},
  title        = {Pith review of: PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YT7ANEQN}},
  note         = {Machine review of arXiv:2607.22695}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) are capable of generalizing human language for the completion of never-before-seen tasks, leading to widespread deployment. While this automation provides clear utility, completing these tasks often requires the insertion of Personally Identifiable Information (PII), strings of information that uniquely identify some individual, raising privacy concerns. However, ethics has prevented the curation of a public, authentic dataset of PII. Without an appropriate dataset, it is difficult to quantify privacy risks. Thus, we introduce the PANOPTICON pipeline and dataset. The dataset, generated by Meta's Llama-3.1-8B-Instruct model, contains 67, 718 prompts, intended for the models context window, containing PII spans derived from 9,674 publicly available synthetic user profiles. We measure lexical diversity and S-BERT diversity of the created dataset to evaluate realism. Finally, we present a case study showcasing the utility of PANOPTICON data for understanding Prompt Inversion Attacks (PIAs). PANOPTICON thus emerges as the first benchmark dataset for studying PIAs over private corpora, providing a foundation for future LLM privacy research.

Figures

Figures reproduced from arXiv: 2607.22695 by Maanak Gupta, Mir Mehedi Ahsan Pritom, Ryan Thornton.

Figure 1
Figure 1. Figure 1: PANOPTICON Dataset Generation Pipeline future directions. II. BACKGROUND AND MOTIVATION In this section, we provide the background research and motivation for this work. A. Large Language Models (LLMs) LLMs come in three architectures: Encoder, Decoder, and hybrid models [10]. Regardless of architecture, the model accepts as input a sequence of tokens representing human language. The training objective is … view at source ↗
Figure 2
Figure 2. Figure 2: PCA projection of Sentence-BERT embeddings for all PANOPTICON [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: PII type distribution across PANOPTICON dataset prompts [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 11 linked inside Pith

  1. [1]

    A survey of large language models,

    W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y . Hou, Y . Min, B. Zhang, J. Zhang, Z. Donget al., “A survey of large language models,”arXiv preprint arXiv:2303.18223, vol. 1, no. 2, pp. 1–124, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Agentic retrieval-augmented generation: A survey on agentic rag,

    A. Singh, A. Ehtesham, S. Kumar, T. T. Khoei, and A. V . Vasilakos, “Agentic retrieval-augmented generation: A survey on agentic rag,”

  3. [3]

    A survey on large language models for critical societal domains: Finance, healthcare, and law,

    Z. Z. Chen, J. Ma, X. Zhang, N. Hao, A. Yan, A. Nourbakhsh, X. Yang, J. McAuley, L. Petzold, and W. Y . Wang, “A survey on large language models for critical societal domains: Finance, healthcare, and law,” 2024. [Online]. Available: https://arxiv.org/abs/2405.01769

    Pith/arXiv arXiv 2024

  4. [4]

    The right to privacy,

    S. D. Warren and L. D. Brandeis, “The right to privacy,”Harvard Law Review, vol. 4, no. 5, pp. 193–220, December 1890

  5. [5]

    Position: Privacy is not just memorization!

    N. Mireshghallah and T. Li, “Position: Privacy is not just memorization!”

  6. [6]

    Panorama: A synthetic pii-laced dataset for studying sensitive data memorization in llms,

    S. Selvam and A. Ghosh, “Panorama: A synthetic pii-laced dataset for studying sensitive data memorization in llms,”arXiv preprint arXiv:2505.12238, 2025, accessed: 2025-12-21. [Online]. Available: https://arxiv.org/abs/2505.12238

    Pith/arXiv arXiv 2025

  7. [7]

    Beyond memoriza- tion: Violating privacy via inference with large language models,

    R. Staab, M. Vero, M. Balunovic, and M. Vechev, “Beyond memoriza- tion: Violating privacy via inference with large language models,” in International Conference on Learning Representations, vol. 2024, 2024, pp. 33 832–33 878

    2024

  8. [8]

    Dory: Deliberative prompt recovery for llm,

    L. Gao, R. Peng, Y . Zhang, and J. Zhao, “Dory: Deliberative prompt recovery for llm,”arXiv preprint arXiv:2405.20657, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Can llms keep a secret? testing privacy implications of language models via contextual integrity theory,

    N. Mireshghallah, H. Kim, X. Zhou, Y . Tsvetkov, M. Sap, R. Shokri, and Y . Choi, “Can llms keep a secret? testing privacy implications of language models via contextual integrity theory,” 2024. [Online]. Available: https://arxiv.org/abs/2310.17884

    Pith/arXiv arXiv 2024

  10. [10]

    How powerful are decoder-only transformer neural models?

    J. Roberts, “How powerful are decoder-only transformer neural models?” in2024 International Joint Conference on Neural Networks (IJCNN). IEEE, June 2024, p. 1–8. [Online]. Available: http://dx.doi.org/10.1109/IJCNN60899.2024.10651286

    arXiv 2024

  11. [11]

    Scaling laws for neural language models,

    J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,”ArXiv, vol. abs/2001.08361, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:210861095

    Pith/arXiv arXiv 2001

  12. [12]

    The Pile: An 800gb dataset of diverse text for language modeling,

    L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, S. Presser, and C. Leahy, “The Pile: An 800gb dataset of diverse text for language modeling,” arXiv preprint arXiv:2101.00027, 2020

    Pith/arXiv arXiv 2020

  13. [13]

    The secret sharer: Evaluating and testing unintended memorization in neural networks,

    N. Carlini, C. Liu, ´U. Erlingsson, J. Kos, and D. Song, “The secret sharer: Evaluating and testing unintended memorization in neural networks,” in28th USENIX Security Symposium (USENIX Security 19). USENIX Association, 2019, pp. 267–284, accessed: 2025-12-21. [Online]. Available: https://www.usenix.org/conference/usenixsecurity19/presentation/carlini

    2019

  14. [14]

    Extracting training data from large language models,

    N. Carlini, F. Tram `er, E. Wallace, M. Jagielski, A. Herbert-V oss, K. Lee, A. Roberts, T. Brown, D. Song, ´U. Erlingsson, A. Oprea, and C. Raffel, “Extracting training data from large language models,” in30th USENIX Security Symposium (USENIX Security 21). USENIX Association, August 2021, pp. 2633–2650. [Online]. Available: https://www.usenix.org/confer...

    2021

  15. [15]

    Quantifying memorization across neural language models,

    N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang, “Quantifying memorization across neural language models,” 2023. [Online]. Available: https://arxiv.org/abs/2202.07646

    Pith/arXiv arXiv 2023

  16. [16]

    Membership inference attacks against machine learning models,

    R. Shokri, M. Stronati, C. Song, and V . Shmatikov, “Membership inference attacks against machine learning models,” in2017 IEEE Symposium on Security and Privacy (SP), 2017, pp. 3–18, accessed: 2025-12-21

    2017

  17. [17]

    Differentially private decoding in large language models,

    J. Majmudar, C. Dupuy, C. Peris, S. Smaili, R. Gupta, and R. Zemel, “Differentially private decoding in large language models,” 2022. [Online]. Available: https://arxiv.org/abs/2205.13621

    Pith/arXiv arXiv 2022

  18. [18]

    Detecting personal information in training corpora: an analysis,

    N. Subramani, A. S. Luccioni, J. Dodge, and M. Mitchell, “Detecting personal information in training corpora: an analysis,” inProceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023). Association for Computational Linguistics, Jul. 2023, pp. 208–220. [Online]. Available: https://aclanthology.org/2023.trustnlp- 1.18/

    2023

  19. [19]

    Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy,

    N. Dowlin, R. Gilad-Bachrach, K. Laine, K. Lauter, M. Naehrig, and J. Wernsing, “Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy,” inProceedings of the 33rd International Conference on Machine Learning (ICML), ser. Proceedings of Machine Learning Research, vol. 48, 2016. [Online]. Available: https://proceedings.mlr...

    2016

  20. [20]

    Dual-priv pruning : Efficient differential private fine-tuning in multimodal large language models,

    Q. Wei, J. Li, Z. You, Y . Zhan, K. Li, J. Wu, X. L. H. Liu, Y . Yu, B. Cao, Y . Xu, Y . Liu, and G. Qi, “Dual-priv pruning : Efficient differential private fine-tuning in multimodal large language models,”

  21. [21]

    An evaluation study of hybrid methods for multilingual pii detection,

    H. Rajgarhia, S. Gupta, A. Shaik, G. P. Kumar, Y . Santhoshraj, S. N. T. Nishitha, and A. Mukherji, “An evaluation study of hybrid methods for multilingual pii detection,” 2025. [Online]. Available: https://arxiv.org/abs/2510.07551

    arXiv 2025

  22. [22]

    EncryptedLLM: Privacy-preserving large language model inference via GPU-accelerated fully homomorphic encryption,

    L. de Castro, D. Escudero, A. Agrawal, A. Polychroniadou, and M. Veloso, “EncryptedLLM: Privacy-preserving large language model inference via GPU-accelerated fully homomorphic encryption,” in Forty-second International Conference on Machine Learning, 2025. [Online]. Available: https://openreview.net/forum?id=PGNff6H1TV

    2025

  23. [23]

    Available: https://arxiv.org/abs/2506.07077

    [Online]. Available: https://arxiv.org/abs/2506.07077

    Pith/arXiv arXiv

  24. [24]

    Effective prompt extraction from language models,

    Y . Zhang, N. Carlini, and D. Ippolito, “Effective prompt extraction from language models,” inConference on Language Modeling (COLM), 2024. APPENDIX A. Initialization Prompts We provide the following role-playing instruction to LLM: “You are a synthetic Personally Identifiable Information (PII) laden prompt generator. Your task is to use the provided synt...

    2024

  25. [26]

    Prompt inversion attack against collaborative inference of large language mod- els,

    W. Qu, Y . Zhou, Y . Wu, T. Xiao, B. Yuan, Y . Li, and J. Zhang, “Prompt inversion attack against collaborative inference of large language mod- els,” in2025 IEEE Symposium on Security and Privacy (SP), 2025

    2025

  26. [28]

    Obtain a continuous token-level representation{z i}L i=1 (lengthL)

  27. [29]

    Project eachz i to a discrete token via nearest-neighbor lookup in embedding space (Appendix D)

  28. [30]

    This setting isolates theprojection stepand avoids any addi- tional search over the discrete space

    Detokenize the resulting token sequence into text and report metrics. This setting isolates theprojection stepand avoids any addi- tional search over the discrete space. b) Refine On.:When refinement is enabled, we apply an additional local search procedure over the discrete sequence to mitigate common failure modes of one-shot projection (e.g., malformed...

  29. [31]

    refine off

    Initialize with the projected token sequence ˆt= (ˆt1, . . . ,ˆtL)from the “refine off” procedure

  30. [32]

    For each positioni, retrieve a small candidate listC i (top-Ktokens) using cosine similarity betweenz i and vocabulary embeddings (Appendix D)

  31. [33]

    Score candidate replacements under a local objective. In practice, we use a context-conditional language-model score (negative log-likelihood / perplexity proxy) com- puted for the reconstructed sequence in a limited window around positioni

  32. [34]

    Accept a replacement ˆti ←cif it improves the objective by a margin∆; otherwise keep ˆti unchanged

  33. [35]

    Refine on

    Repeat for up toRrefinement rounds or until no positions accept updates (early stopping). Rationale.“Refine on” adds a small amount of discrete exploration while keeping computation bounded (controlled byKcandidates per position andRrounds). It is particularly helpful for PII-heavy strings (emails, phone numbers, IDs), where tokenization can be brittle an...

  34. [2025]

    Available: https://arxiv.org/abs/2510.01645

    [Online]. Available: https://arxiv.org/abs/2510.01645

    arXiv

  35. [2026]

    Available: https://arxiv.org/abs/2501.09136

    [Online]. Available: https://arxiv.org/abs/2501.09136

    Pith/arXiv arXiv