pith. sign in

arxiv: 2507.02974 · v3 · submitted 2025-06-30 · 💻 cs.LG · cs.CL· cs.CR

InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy

Vishnu Vinod , Krishna Pillutla , Abhradeep Guha Thakurta This is my paper

Pith reviewed 2026-05-19 06:47 UTC · model grok-4.3

classification 💻 cs.LG cs.CLcs.CR
keywords differential privacytext generationlarge language modelslong-form generationcomputational efficiencyexponential mechanismlogit clipping
0
0 comments X

The pith

InvisibleInk generates high-quality private long-form text at 4-8 times the cost of non-private generation by clipping only the sensitive parts of model logits.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces InvisibleInk as a framework that adds differential privacy to long-form text generation from large language models. It does so by separating the model's output logits into public and private components, then applying clipping and noise only to the private ones while sampling top private tokens at no extra privacy cost. This design cuts computation costs by a factor of eight or more compared with earlier private methods. A reader would care because it brings rigorous privacy protection within reach for practical uses like retrieval-augmented generation without requiring enormous extra compute. If correct, the approach makes it feasible to produce useful private text at overhead low enough for real applications.

Core claim

InvisibleInk treats next-token sampling as the exponential mechanism over LLM logits. It reduces privacy cost by isolating and clipping only the sensitive information in the logits relative to the public logits. It further improves quality by sampling without privacy cost from a small superset of the top-k private tokens. These steps together deliver an 8x or greater drop in computation cost versus prior private baselines and, for the first time, high-quality private long-form text at less than 4-8x the cost of ordinary non-private generation.

What carries the argument

Separation of LLM logits into public and private components so that clipping and noise are applied only to the private portion, together with privacy-free sampling from a small superset of top-k private tokens.

If this is right

  • Delivers consistent 8x or greater reduction in computation cost over state-of-the-art private text generation baselines at the same utility level.
  • Produces long-form private text whose quality approaches that of non-private generation while satisfying rigorous differential privacy with respect to sensitive references.
  • Supports safe incorporation of private information into retrieval-augmented generation and inference-time scaling workflows.
  • Enables the first practical regime in which high-utility private long-form text can be generated at modest overhead relative to ordinary sampling.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same logit-isolation idea could be tested on other autoregressive tasks such as code completion if analogous public-private decompositions exist.
  • Running the method on datasets with highly correlated private references would test whether the claimed separation remains stable in realistic settings.
  • Combining the approach with existing speed-ups such as speculative decoding might push the overhead even closer to non-private levels.

Load-bearing premise

The model's logits can be cleanly split into parts driven only by public data and parts driven by private reference texts so that noise on the private part alone still yields high-quality output while preserving the privacy guarantee.

What would settle it

An experiment that shows either a large drop in generated-text quality after isolating and noising only the private logits or a privacy violation when the decomposition is used on real sensitive reference texts.

Figures

Figures reproduced from arXiv: 2507.02974 by Abhradeep Guha Thakurta, Krishna Pillutla, Vishnu Vinod.

Figure 1
Figure 1. Figure 1: INVISIBLEINK interprets differentially private text generation as an iterative application of the exponential mechanism over a subset of the LLM’s clipped logits. Our key innovations are: (a) DClip, an improved clipping function to reduce the sensitivity, and hence, the privacy cost; and (b) Top-k+ sampling, a truncated decoding algorithm to improve utility by cleverly selecting a subset of logits to sampl… view at source ↗
Figure 2
Figure 2. Figure 2: Left & Center: Illustration of how two common decoding algorithms—temperature rescaling and top-k sampling—reshape the next-token probabilities. Right: Heatmap of MAUVE scores [16, 17] of synthetic text generated for the MIMIC-IV dataset (without using any sensitive references). Notice that the best generations (highest MAUVE) are obtained at τ ≈ 1 and k ≈ 100; INVISIBLEINK exhibits similar behavior. 3 Pre… view at source ↗
Figure 3
Figure 3. Figure 3: Left two: Histograms of private logits ϕi and differences from public logits ϕi − ϕpub for a synthetic data sample generated from the MIMIC dataset, with 5 th and 95th percentiles shown by the dotted lines. The spread of values for ϕi − ϕpub is significantly smaller (around 10×) than that of ϕi . We find that DClipC (ϕi , ϕpub)(y) = ϕi(y) for over 95% of all y ∈ V with C ≈ 1, while the naive clipping of Am… view at source ↗
Figure 4
Figure 4. Figure 4: Utility-compute tradeoffs at (ε = 10, δ = 10−6 ) DP on each dataset across varying compute budget from B ∈ {2, . . . , 128}. INVISIBLEINK can produce text that matches or exceeds the baselines at a fraction of the compute. The baselines do not even work for the low-resource TAB dataset at a small batch size B = 8. procedures, and medicines. We plot the number of such entities identified by the MedNER model… view at source ↗
Figure 5
Figure 5. Figure 5: Utility vs Computational Cost plots for INVISIBLEINK and AugPE for ε = 10 for 1000 synthetic texts generated for the MIMIC dataset. We compare utility using a variety of metrics — INVISIBLEINK outperforms AugPE across all settings and evaluation metrics. Wall-clock run time is used as a proxy for computational cost. We report results for B + 1 = 4, 8, 16, 32 next-token inference calls per generated token a… view at source ↗
Figure 6
Figure 6. Figure 6: Truncated decoding offers significant benefits: MAUVE scores for DP synthetic text generation of 1000 samples from the MIMIC Dataset for various privacy budgets using TinyLLaMA-1B and LLaMA3.2-1B models for various top-k thresholds at a temperature τ = 1.2 and batch size B = 7. generations of INVISIBLEINK. We found in our preliminary experiments that AugPE showed competitive (with INVISIBLEINK) performance… view at source ↗
Figure 7
Figure 7. Figure 7: Utility vs Computational Cost plots for INVISIBLEINK and AugPE for varying privacy budgets ε for 1000 synthetic texts generated for the MIMIC dataset. We compare utility using a variety of metrics — INVISIBLEINK outperforms AugPE across all settings and evaluation metrics. Wall-clock run time is used to measure the computational cost. We report results for B + 1 = 4, 8, 16 next-token inference calls per ge… view at source ↗
Figure 8
Figure 8. Figure 8: Variation of utility (measured using a variety of metrics) with sampling temperature, for the full-vocabulary variant of INVISIBLEINK (k = |V |), reported for 1000 synthetic generations for the MIMIC dataset using a TinyLLaMA 1B model. Temperature is varied from 0.8 − 1.1 for a fixed privacy budget ε = 10. We observe that selecting a higher temperature consistently gives better performance [PITH_FULL_IMAG… view at source ↗
Figure 9
Figure 9. Figure 9: Effect of clipping norm. Left two: Variation of calculated C for INVISIBLEINK and Amin et al. [12]’s method with batch size B for various privacy budgets. The latter needs much larger batch sizes to give comparable clip norms for a given privacy budget. Right two: Variation of utility with C for 1000 synthetic generations for the MIMIC dataset using a TinyLLaMA 1B model using INVISIBLEINK (k = |V |). Tempe… view at source ↗
read the original abstract

As major progress in LLM-based long-form text generation enables paradigms such as retrieval-augmented generation (RAG) and inference-time scaling, safely incorporating private information into the generation remains a critical open question. We present InvisibleInk, a highly scalable long-form text generation framework satisfying rigorous differential privacy guarantees with respect to the sensitive reference texts. It interprets sampling from the LLM's next-token-distribution as the exponential mechanism over the LLM logits with two innovations. First, we reduce the privacy cost by isolating and clipping only the sensitive information in the model logits (relative to the public logits). Second, we improve text quality by sampling without any privacy cost from a small superset of the top-$k$ private tokens. Empirical evaluations demonstrate a consistent $8\times$ (or more) reduction in computation cost over state-of-the-art baselines to generate long-form private text of the same utility across privacy levels. InvisibleInk is able to generate, for the first time, high-quality private long-form text at less than $4$-$8\times$ times the computation cost of non-private generation, paving the way for its practical use. We open-source a pip-installable Python package (invink) for InvisibleInk at https://github.com/cerai-iitm/invisibleink.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces InvisibleInk, a framework for differentially private long-form text generation using LLMs. It interprets next-token sampling as the exponential mechanism over logits and proposes two innovations: isolating and clipping only the sensitive information in logits relative to public logits to reduce privacy cost, and sampling without privacy cost from a small superset of the top-k private tokens to improve quality. The central empirical claim is a consistent 8× (or more) reduction in computation cost over state-of-the-art baselines while achieving comparable utility, enabling high-quality private long-form text at 4-8× the cost of non-private generation. The work also releases an open-source Python package.

Significance. If the logit decomposition rigorously satisfies differential privacy and the reported cost-utility tradeoffs are reproducible, the result would be significant for enabling practical private text generation in settings such as RAG and inference-time scaling. The open-sourcing of the invink package supports reproducibility and is a positive contribution.

major comments (2)
  1. [§3] §3 (Method, exponential mechanism interpretation): The central innovation of clipping only the 'sensitive information' in logits relative to public logits lacks a formal definition of the public/private split and a proof that the resulting mechanism satisfies standard differential privacy (e.g., with respect to a single reference text change). This decomposition is load-bearing for both the claimed privacy guarantee and the 4-8× cost reduction; without it, the separation appears heuristic and risks either leaking information through the public component or invalidating the DP bound.
  2. [§5] §5 (Empirical evaluations): The abstract and claims assert an 8× cost reduction and high utility across privacy levels with no experimental details, datasets, baseline implementations, or statistical reporting visible in the manuscript text. This undermines verification of the cross-baseline and cross-privacy-level claims.
minor comments (1)
  1. [§3] The notation for the top-k superset size and sensitive logit clipping threshold should be explicitly defined with symbols and ranges in the method section to aid reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their detailed and constructive review. We address each major comment below and have revised the manuscript accordingly to strengthen the formal foundations and empirical presentation.

read point-by-point responses

  1. Referee: [§3] §3 (Method, exponential mechanism interpretation): The central innovation of clipping only the 'sensitive information' in logits relative to public logits lacks a formal definition of the public/private split and a proof that the resulting mechanism satisfies standard differential privacy (e.g., with respect to a single reference text change). This decomposition is load-bearing for both the claimed privacy guarantee and the 4-8× cost reduction; without it, the separation appears heuristic and risks either leaking information through the public component or invalidating the DP bound.

    Authors: We agree that a rigorous formalization is essential for the claimed privacy guarantees. In the revised manuscript, Section 3.1 now provides an explicit definition: public logits are those produced by an LLM fine-tuned exclusively on public data, while the sensitive component is the difference (delta) between logits from the full model (including private reference texts) and the public logits. We have added a complete proof in Appendix A showing that the mechanism—clipping only this sensitive delta to bound sensitivity, then applying the exponential mechanism—satisfies (ε, δ)-DP with respect to replacement of a single reference text. The proof proceeds by bounding the log-ratio of probabilities under neighboring datasets after clipping and shows that the subsequent top-k superset sampling incurs no additional privacy cost because it is post-processing. These additions directly support both the privacy claim and the reported cost savings. revision: yes

  2. Referee: [§5] §5 (Empirical evaluations): The abstract and claims assert an 8× cost reduction and high utility across privacy levels with no experimental details, datasets, baseline implementations, or statistical reporting visible in the manuscript text. This undermines verification of the cross-baseline and cross-privacy-level claims.

    Authors: We acknowledge that the experimental details were not presented with sufficient prominence or completeness in the original text. The revised Section 5 now includes: (i) explicit dataset descriptions (including the specific long-form generation benchmarks and RAG-style tasks used), (ii) implementation details for all baselines with references to the original papers and our reproduction choices, (iii) full hyperparameter settings and privacy budgets (ε values), and (iv) statistical reporting with means and standard deviations computed over five independent runs. A new summary table has been added that directly quantifies the 8× (or greater) wall-clock and FLOPs reduction relative to baselines at matched utility levels across privacy regimes. These changes make the empirical claims verifiable from the manuscript text. revision: yes

Circularity Check

0 steps flagged

Minor self-citation present but not load-bearing; core derivation self-contained

full rationale

The paper interprets next-token sampling as the exponential mechanism and proposes two innovations for clipping sensitive logit components and free sampling from top-k supersets. These steps are presented as novel algorithmic choices rather than reductions to fitted parameters or prior self-citations. No equation or claim equates the DP guarantee or utility claim directly to its own inputs by construction. Standard DP primitives and LLM sampling mechanics provide independent grounding, with empirical results offering external validation. A single minor self-citation (if present in related DP work) does not carry the central claims.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The central claim rests on the standard interpretation of next-token sampling as an exponential mechanism and the domain assumption that logits can be partitioned into public and private components. No new entities are postulated. Free parameters such as the top-k size and clipping threshold are implicit but not quantified in the abstract.

free parameters (2)
  • top-k superset size
    Small superset of top private tokens chosen to trade off quality against efficiency; value not reported in abstract.
  • sensitive logit clipping threshold
    Bound used to control sensitivity for the privacy mechanism; value not reported in abstract.
axioms (1)
  • domain assumption Next-token sampling from an LLM can be interpreted as the exponential mechanism over the model logits.
    This is the explicit starting point stated in the abstract for applying differential privacy.

pith-pipeline@v0.9.0 · 5768 in / 1403 out tokens · 53516 ms · 2026-05-19T06:47:58.094051+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Differentially Private Sampling from Distributions via Wasserstein Projection

    stat.ML 2026-05 unverdicted novelty 7.0

    Proposes Wasserstein Projection Mechanism for differentially private sampling that optimizes Wasserstein distance utility and provides convergence guarantees for approximate computation.

Reference graph

Works this paper leans on

90 extracted references · 90 canonical work pages · cited by 1 Pith paper · 3 internal anchors

  1. [1]

    Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving

    Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving. In ICLR, 2025

    work page 2025

  2. [2]

    Candès, and Tatsunori Hashimoto

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel J. Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. Preprint, 2025

    work page 2025

  3. [3]

    Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead

    Vidhisha Balachandran, Jingya Chen, Lingjiao Chen, Shivam Garg, Neel Joshi, Yash Lara, John Langford, Besmira Nushi, Vibhav Vineet, Yue Wu, et al. Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead. Preprint, 2025

    work page 2025

  4. [4]

    Maxwell I. Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. Show Your Work: Scratchpads for Intermediate Computation with Language Models. Preprint, 2021

    work page 2021

  5. [5]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS, 35: 24824–24837, 2022

    work page 2022

  6. [6]

    STaR: Self-Taught Reasoner Bootstrap- ping Reasoning With Reasoning

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D Goodman. STaR: Self-Taught Reasoner Bootstrap- ping Reasoning With Reasoning. In NeurIPS, volume 1126, 2022

    work page 2022

  7. [7]

    Tree of Thoughts: Deliberate Problem Solving with Large Language Models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. In NeurIPS, 2023. 14

    work page 2023

  8. [8]

    From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models

    Sean Welleck, Amanda Bertsch, Matthew Finlayson, Hailey Schoelkopf, Alex Xie, Graham Neubig, Ilia Kulikov, and Zaid Harchaoui. From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models. TMLR, 2024

    work page 2024

  9. [9]

    Scalable Extraction of Training Data from (Production) Language Models

    Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:2311.17035, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  10. [10]

    The Canary’s Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text.Preprint, 2025

    Matthieu Meeus, Lukas Wutschitz, Santiago Zanella-Béguelin, Shruti Tople, and Reza Shokri. The Canary’s Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text.Preprint, 2025

    work page 2025

  11. [11]

    Calibrating Noise to Sensitivity in Private Data Analysis

    Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating Noise to Sensitivity in Private Data Analysis. In Proc. of the Third Conf. on Theory of Cryptography (TCC), pages 265–284,

    work page

  12. [12]

    URL http://dx.doi.org/10.1007/11681878_14

    work page doi:10.1007/11681878_14

  13. [13]

    Private prediction for large-scale synthetic text generation, 2024

    Kareem Amin, Alex Bie, Weiwei Kong, Alexey Kurakin, Natalia Ponomareva, Umar Syed, Andreas Terzis, and Sergei Vassilvitskii. Private prediction for large-scale synthetic text generation, 2024. URL https://arxiv.org/abs/2407.12108

    work page arXiv 2024

  14. [14]

    In: 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS ’07), pp

    Frank McSherry and Kunal Talwar. Mechanism design via differential privacy. InFOCS, pages 94–103, 11 2007. ISBN 978-0-7695-3010-9. doi: 10.1109/FOCS.2007.66

    work page doi:10.1109/focs.2007.66 2007

  15. [15]

    How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy

    Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin, Zheng Xu, Carson Denison, H Brendan McMa- han, Sergei Vassilvitskii, Steve Chien, and Abhradeep Guha Thakurta. How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy. Journal of Artificial Intelligence Research, 77: 1113–1201, 2023

    work page 2023

  16. [16]

    The Curious Case of Neural Text Degeneration

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The Curious Case of Neural Text Degeneration. In ICLR, 2020

    work page 2020

  17. [17]

    Mauve: Measuring the gap between neural text and human text using divergence frontiers

    Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. Mauve: Measuring the gap between neural text and human text using divergence frontiers. In NeurIPS, 2021

    work page 2021

  18. [18]

    MAUVE Scores for Generative Models: Theory and Practice

    Krishna Pillutla, Lang Liu, John Thickstun, Sean Welleck, Swabha Swayamdipta, Rowan Zellers, Sewoong Oh, Yejin Choi, and Zaid Harchaoui. MAUVE Scores for Generative Models: Theory and Practice. JMLR, 2023

    work page 2023

  19. [19]

    Adaptively private next-token prediction of large language models

    James Flemings, Meisam Razaviyayn, and Murali Annavaram. Adaptively private next-token prediction of large language models. arXiv preprint arXiv:2410.02016, 2024

    work page arXiv 2024

  20. [20]

    Hyperparameter Tuning with Renyi Differential Privacy

    Nicolas Papernot and Thomas Steinke. Hyperparameter Tuning with Renyi Differential Privacy. In ICLR, 2022

    work page 2022

  21. [21]

    A learning theory approach to non-interactive database privacy

    Avrim Blum, Katrina Ligett, and Aaron Roth. A learning theory approach to non-interactive database privacy. In Cynthia Dwork, editor, STOC, pages 609–618, 2008

    work page 2008

  22. [22]

    Rothblum

    Moritz Hardt and Guy N. Rothblum. A Multiplicative Weights Mechanism for Privacy-Preserving Data Analysis. In FOCS, pages 61–70. IEEE Computer Society, 2010

    work page 2010

  23. [23]

    Rothblum, and Salil P

    Cynthia Dwork, Moni Naor, Omer Reingold, Guy N. Rothblum, and Salil P. Vadhan. On the complexity of differentially private data release: efficient algorithms and hardness results. In STOC, pages 381–390, 2009. 15

    work page 2009

  24. [24]

    Private Synthetic Data for Multitask Learning and Marginal Queries

    Giuseppe Vietri, Cédric Archambeau, Sergül Aydöre, William Brown, Michael Kearns, Aaron Roth, Amaresh Ankit Siva, Shuai Tang, and Zhiwei Steven Wu. Private Synthetic Data for Multitask Learning and Marginal Queries. In NeurIPS, 2022

    work page 2022

  25. [25]

    Online Differentially Private Synthetic Data Generation

    Yiyun He, Roman Vershynin, and Yizhe Zhu. Online Differentially Private Synthetic Data Generation. IEEE Transactions on Privacy, 2024

    work page 2024

  26. [26]

    A Simple and Practical Algorithm for Differentially Private Data Release

    Moritz Hardt, Katrina Ligett, and Frank McSherry. A Simple and Practical Algorithm for Differentially Private Data Release. NeurIPS, 25, 2012

    work page 2012

  27. [27]

    On the Gibbs Exponential Mechanism and Private Synthetic Data Generation

    Amir-Reza Asadi and Po-Ling Loh. On the Gibbs Exponential Mechanism and Private Synthetic Data Generation. In ISIT, pages 2213–2218. IEEE, 2023

    work page 2023

  28. [28]

    PreFair: Privately Generating Justifiably Fair Synthetic Data

    David Pujol, Amir Gilad, and Ashwin Machanavajjhala. PreFair: Privately Generating Justifiably Fair Synthetic Data. Proc. VLDB Endow., 16(6):1573–1586, 2023

    work page 2023

  29. [29]

    Ullman and Salil P

    Jonathan R. Ullman and Salil P. Vadhan. PCPs and the Hardness of Generating Private Synthetic Data. In Theory of Cryptography Conference, volume 6597 of Lecture Notes in Computer Science, pages 400–416. Springer, 2011

    work page 2011

  30. [30]

    Salil P. Vadhan. The Complexity of Differential Privacy. InTutorials on the Foundations of Cryptogra- phy, pages 347–450. Springer International Publishing, 2017

    work page 2017

  31. [31]

    2014.The Algorithmic Foundations of Differential Privacy

    Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9(3–4):211–407, 2014. ISSN 1551-305X. doi: 10.1561/ 0400000042. URL http://dx.doi.org/10.1561/0400000042

    work page doi:10.1561/0400000042 2014

  32. [32]

    Differentially Private Speaker Anonymization

    Ali Shahin Shamsabadi, Brij Mohan Lal Srivastava, Aurélien Bellet, Nathalie Vauquier, Emmanuel Vincent, Mohamed Maouche, Marc Tommasi, and Nicolas Papernot. Differentially Private Speaker Anonymization. Proc. Priv. Enhancing Technol., 2023(1):98–114, 2023

    work page 2023

  33. [33]

    Differentially Private Diffusion Models

    Tim Dockhorn, Tianshi Cao, Arash Vahdat, and Karsten Kreis. Differentially Private Diffusion Models. Trans. Mach. Learn. Res., 2023

    work page 2023

  34. [34]

    PrivImage: Differ- entially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretraining

    Kecen Li, Chen Gong, Zhixiang Li, Yuzhong Zhao, Xinwen Hou, and Tianhao Wang. PrivImage: Differ- entially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretraining. In USENIX Security Symposium. USENIX Association, 2024

    work page 2024

  35. [35]

    Hermite Polynomial Features for Private Data Generation

    Margarita Vinaroz, Mohammad-Amin Charusaie, Frederik Harder, Kamil Adamczewski, and Mijung Park. Hermite Polynomial Features for Private Data Generation. In ICML, volume 162, pages 22300– 22324. PMLR, 2022

    work page 2022

  36. [36]

    Differentially Private Synthetic Data via Foundation Model APIs 1: Images

    Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, and Sergey Yekhanin. Differentially Private Synthetic Data via Foundation Model APIs 1: Images. In ICLR, 2024

    work page 2024

  37. [37]

    Differentially Private Synthetic Data via Foundation Model APIs 2: Text, 2024

    Chulin Xie, Zinan Lin, Arturs Backurs, Sivakanth Gopi, Da Yu, Huseyin A Inan, Harsha Nori, Haotian Jiang, Huishuai Zhang, Yin Tat Lee, Bo Li, and Sergey Yekhanin. Differentially Private Synthetic Data via Foundation Model APIs 2: Text, 2024. URL https://arxiv.org/abs/2403.01749

    work page arXiv 2024

  38. [38]

    Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, and Blaise Agüera y Arcas

    Sean Augenstein, H. Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, and Blaise Agüera y Arcas. Generative Models for Effective ML on Private, Decentralized Datasets. In ICLR, 2020. 16

    work page 2020

  39. [39]

    Differentially Private Language Models for Secure Data Sharing

    Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schölkopf, and Mrinmaya Sachan. Differentially Private Language Models for Secure Data Sharing. In EMNLP, pages 4860–4873. Association for Computational Linguistics, 2022

    work page 2022

  40. [40]

    Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim

    Xiang Yue, Huseyin A. Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim. Synthetic text generation with differential privacy: A simple and practical recipe, 2023. URL https://arxiv.org/abs/2210.14348

    work page arXiv 2023

  41. [41]

    Harnessing large-language models to generate private synthetic text, 2024

    Alexey Kurakin, Natalia Ponomareva, Umar Syed, Liam MacDermed, and Andreas Terzis. Harnessing large-language models to generate private synthetic text, 2024. URL https://arxiv.org/abs/2306. 01684

    work page 2024

  42. [42]

    Privacy-preserving instructions for aligning large language models, 2024

    Da Yu, Peter Kairouz, Sewoong Oh, and Zheng Xu. Privacy-preserving instructions for aligning large language models, 2024. URL https://arxiv.org/abs/2402.13659

    work page arXiv 2024

  43. [43]

    Wang, and Prateek Mittal

    Tong Wu, Ashwinee Panda, Jiachen T. Wang, and Prateek Mittal. Privacy-preserving in-context learning for large language models, 2023. URL https://arxiv.org/abs/2305.01639

    work page arXiv 2023

  44. [44]

    Inan, Andre Manoel, Fatemehsadat Mireshghallah, Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, and Robert Sim

    Xinyu Tang, Richard Shin, Huseyin A. Inan, Andre Manoel, Fatemehsadat Mireshghallah, Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, and Robert Sim. Privacy-preserving in-context learning with differentially private few-shot generation, 2024. URL https://arxiv.org/abs/2309.11765

    work page arXiv 2024

  45. [45]

    TEM: High Utility Metric Differential Privacy on Text

    Ricardo Silva Carvalho, Theodore Vasiloudis, Oluwaseyi Feyisetan, and Ke Wang. TEM: High Utility Metric Differential Privacy on Text. In SIAM International Conference on Data Mining, pages 883–890. SIAM, 2023

    work page 2023

  46. [46]

    Locally Differentially Private Document Generation Using Zero Shot Prompting

    Saiteja Utpala, Sara Hooker, and Pin-Yu Chen. Locally Differentially Private Document Generation Using Zero Shot Prompting. In EMNLP Findings, pages 8442–8457. Association for Computational Linguistics, 2023

    work page 2023

  47. [47]

    Jimit Majmudar, Christophe Dupuy, Charith Peris, Sami Smaili, Rahul Gupta, and Richard S. Zemel. Differentially Private Decoding in Large Language Models. Preprint, 2022

    work page 2022

  48. [48]

    Submix: Practical Private Prediction for Large-Scale Language Models, 2022

    Antonio Ginart, Laurens van der Maaten, James Zou, and Chuan Guo. Submix: Practical Private Prediction for Large-Scale Language Models, 2022. URL https://arxiv.org/abs/2201.00971

    work page arXiv 2022

  49. [49]

    Differentially private next-token prediction of large language models, 2024

    James Flemings, Meisam Razaviyayn, and Murali Annavaram. Differentially private next-token prediction of large language models, 2024. URL https://arxiv.org/abs/2403.15638

    work page arXiv 2024

  50. [50]

    Flocks of stochastic parrots: Differentially private prompt learning for large language models, 2023

    Haonan Duan, Adam Dziedzic, Nicolas Papernot, and Franziska Boenisch. Flocks of stochastic parrots: Differentially private prompt learning for large language models, 2023

    work page 2023

  51. [51]

    Wang, Chenhui Zhang, Zhangheng Li, Bo Li, and Zhangyang Wang

    Junyuan Hong, Jiachen T. Wang, Chenhui Zhang, Zhangheng Li, Bo Li, and Zhangyang Wang. Dp-opt: Make large language model your privacy-preserving prompt engineer, 2024. URL https://arxiv. org/abs/2312.03724

    work page arXiv 2024

  52. [52]

    Clustering and median aggregation improve differentially private inference, 2025

    Kareem Amin, Salman Avestimehr, Sara Babakniya, Alex Bie, Weiwei Kong, Natalia Ponomareva, and Umar Syed. Clustering and median aggregation improve differentially private inference, 2025. URL https://arxiv.org/abs/2506.04566

    work page arXiv 2025

  53. [53]

    Accuracy first: Selecting a differential privacy level for accuracy constrained ERM

    Katrina Ligett, Seth Neel, Aaron Roth, Bo Waggoner, and Zhiwei Steven Wu. Accuracy first: Selecting a differential privacy level for accuracy constrained ERM. In Isabelle Guyon, Ulrike 17 von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V . N. Vishwanathan, and Ro- man Garnett, editors, Advances in Neural Information Processing Systems 30: Annua...

    work page 2017

  54. [54]

    Angela Fan, Mike Lewis, and Yann N. Dauphin. Hierarchical Neural Story Generation. In Proc. of ACL, pages 889–898, 2018

    work page 2018

  55. [55]

    Locally Typical Sampling.Transactions of the Association for Computational Linguistics, 2022

    Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. Locally Typical Sampling.Transactions of the Association for Computational Linguistics, 2022

    work page 2022

  56. [56]

    Truncation Sampling as Language Model Desmoothing

    John Hewitt, Christopher D Manning, and Percy Liang. Truncation Sampling as Language Model Desmoothing. In Proc. of EMNLP Findings, 2022

    work page 2022

  57. [57]

    Improving Open-Ended Text Generation via Adaptive Decoding

    Wenhong Zhu, Hongkun Hao, Zhiwei He, Yiming Ai, and Rui Wang. Improving Open-Ended Text Generation via Adaptive Decoding. In ICML, 2024

    work page 2024

  58. [58]

    Roush, Andreas Kirsch, and Ravid Shwartz- Ziv

    Nguyen Nhat Minh, Andrew Baker, Clement Neo, Allen G. Roush, Andreas Kirsch, and Ravid Shwartz- Ziv. Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs. In ICLR, 2025

    work page 2025

  59. [59]

    Closing the Curious Case of Neural Text Degeneration

    Matthew Finlayson, John Hewitt, Alexander Koller, Swabha Swayamdipta, and Ashish Sabharwal. Closing the Curious Case of Neural Text Degeneration. In ICLR, 2024

    work page 2024

  60. [60]

    A learning algorithm for Boltzmann machines

    David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski. A learning algorithm for Boltzmann machines. Cognitive science, 9(1):147–169, 1985

    work page 1985

  61. [61]

    Concentrated differential privacy: Simplifications, extensions, and lower bounds

    Mark Bun and Thomas Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. In Martin Hirt and Adam Smith, editors, Theory of Cryptography, pages 635–658, Berlin, Heidelberg, 2016. Springer Berlin Heidelberg. ISBN 978-3-662-53641-4

    work page 2016

  62. [62]

    Rényi Differential Privacy

    Ilya Mironov. Rényi Differential Privacy. In 30th IEEE Computer Security Foundations Symposium, CSF 2017, Santa Barbara, CA, USA, August 21-25, 2017 , pages 263–275. IEEE Computer Society, 2017

    work page 2017

  63. [63]

    Hypothesis Testing Interpretations and Renyi Differential Privacy

    Borja Balle, Gilles Barthe, Marco Gaboardi, Justin Hsu, and Tetsuya Sato. Hypothesis Testing Interpretations and Renyi Differential Privacy. In The 23rd International Conference on Artificial Intelligence and Statistics, volume 108, pages 2496–2506. PMLR, 2020

    work page 2020

  64. [64]

    A better privacy analysis of the exponential mechanism

    Ryan Rogers and Thomas Steinke. A better privacy analysis of the exponential mechanism. DifferentialPrivacy.org, 07 2021. https://differentialprivacy.org/ exponential-mechanism-bounded-range/

    work page 2021

  65. [65]

    Boosting and Differential Privacy

    Cynthia Dwork, Guy N Rothblum, and Salil Vadhan. Boosting and Differential Privacy. In FOCS, pages 51–60. IEEE, 2010

    work page 2010

  66. [66]

    Gállego, Ioannis Tsiamas, and Marta R

    Javier Ferrando, Gerard I. Gállego, Ioannis Tsiamas, and Marta R. Costa-jussà. Explaining how transformers use context to build predictions. In Proc. ACL, pages 5486–5513, July 2023

    work page 2023

  67. [67]

    Language Models: A Guide for the Perplexed

    Sofia Serrano, Zander Brumbaugh, and Noah A Smith. Language Models: A Guide for the Perplexed. arXiv Preprint, 2023. 18

    work page 2023

  68. [68]

    Eliciting Latent Predictions from Transformers with the Tuned Lens

    Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting Latent Predictions from Transformers with the Tuned Lens. arXiv Preprint, 2023

    work page 2023

  69. [69]

    Private Convex Optimization via Exponential Mechanism

    Sivakanth Gopi, Yin Tat Lee, and Daogao Liu. Private Convex Optimization via Exponential Mechanism. In COLT, pages 1948–1989. PMLR, 2022

    work page 1948

  70. [70]

    Universality of Langevin Diffusion for Private Optimization, with Applications to Sampling from Rashomon Sets

    Arun Ganesh, Abhradeep Thakurta, and Jalaj Upadhyay. Universality of Langevin Diffusion for Private Optimization, with Applications to Sampling from Rashomon Sets. In COLT, pages 1730–1773. PMLR, 2023

    work page 2023

  71. [71]

    Alistair E. W. Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J. Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, Li wei H. Lehman, Leo A. Celi, and Roger G. Mark. MIMIC-IV , a freely accessible electronic health record dataset. Scientific Data, 10, January 2023. ISSN 2052-4463. doi: 10.1038/s41597-022-01899-x. URL http...

    work page doi:10.1038/s41597-022-01899-x 2023

  72. [72]

    The text anonymization benchmark (tab): A dedicated corpus and evaluation framework for text anonymization, 2022

    Ildikó Pilán, Pierre Lison, Lilja Øvrelid, Anthi Papadopoulou, David Sánchez, and Montserrat Batet. The text anonymization benchmark (tab): A dedicated corpus and evaluation framework for text anonymization, 2022. URL https://arxiv.org/abs/2202.00443

    work page arXiv 2022

  73. [73]

    https://business.yelp.com/data/resources/open-dataset/

    Yelp dataset, 2023. https://business.yelp.com/data/resources/open-dataset/

    work page 2023

  74. [74]

    TinyLlama: An Open-Source Small Language Model

    Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. Tinyllama: An open-source small language model, 2024. URL https://arxiv.org/abs/2401.02385

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  75. [75]

    Llama 3.2 model card, 2024

    AI@Meta. Llama 3.2 model card, 2024. URL https://github.com/meta-llama/llama-models/ blob/main/models/llama3_2/MODEL_CARD.md

    work page 2024

  76. [76]

    Llama 3 model card, 2024

    AI@Meta. Llama 3 model card, 2024. URL https://github.com/meta-llama/llama3/blob/ main/MODEL_CARD.md

    work page 2024

  77. [77]

    Divergence Frontiers for Generative Models: Sample Complexity, Quantization Effects, and Frontier Integrals

    Lang Liu, Krishna Pillutla, Sean Welleck, Sewoong Oh, Yejin Choi, and Zaid Harchaoui. Divergence Frontiers for Generative Models: Sample Complexity, Quantization Effects, and Frontier Integrals. In NeurIPS, 2021

    work page 2021

  78. [78]

    DeBERTa-MedNER-2

    Saketh Mattupalli. DeBERTa-MedNER-2. https://huggingface.co/blaze999/Medical-NER, 2024

    work page 2024

  79. [79]

    Choquette-Choo, and Zheng Xu

    Nikhil Kandpal, Krishna Pillutla, Alina Oprea, Peter Kairouz, Christopher A. Choquette-Choo, and Zheng Xu. User inference attacks on large language models. In Proc. of EMNLP, pages 18238–18265. Association for Computational Linguistics, November 2024

    work page 2024

  80. [80]

    Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining

    Florian Tramèr, Gautam Kamath, and Nicholas Carlini. Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining. Preprint, 2022

    work page 2022

Showing first 80 references.