pith. machine review for the scientific record. sign in

arxiv: 2509.14594 · v2 · submitted 2025-09-18 · 💻 cs.AI

SynBench: A Benchmark for Differentially Private Text Generation

Yidan Sun , Viktor Schlegel , Srinivasan Nandakumar , Iqra Zahid , Yuping Wu , Yulong Wu , Hao Li , Jie Zhang
show 4 more authors
Warren Del-Pinto Goran Nenadic Siew Kei Lam Anil Anthony Bharath
This is my paper

Pith reviewed 2026-05-18 16:42 UTC · model grok-4.3

classification 💻 cs.AI
keywords differential privacysynthetic text generationmembership inferencebenchmarklarge language modelsprivacy auditingdata contamination
0
0 comments X

The pith

Public pre-training on data similar to private targets breaks the privacy guarantees of differentially private synthetic text generation.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes SynBench, a unified framework with nine curated datasets, standardized utility and fidelity metrics, and privacy audits to evaluate LLM-based differentially private text generators from 1B to 8B parameters. It demonstrates that synthetic text quality drops sharply as private datasets diverge from the models' pre-training corpora. A new synthetic text membership inference attack reveals the root cause: quality appears higher and privacy bounds appear intact only because models were pre-trained without DP on overlapping portions of the supposedly private data. This finding directly challenges the public pre-training plus private generation paradigm used in practice.

Core claim

The central claim is that synthetic data quality is overestimated when LLMs have been pre-trained without DP on portions of the private data to be generated, which invalidates the guaranteed privacy bounds of real-world private datasets, as shown through large-scale benchmarking on nine domain-specific datasets and a novel membership inference attack that succeeds precisely when pre-training contamination is present.

What carries the argument

The novel synthetic text membership inference attack that detects whether generated samples originate from the private dataset by exploiting pre-training leakage.

If this is right

  • Quality of DP-generated text deteriorates more severely when private datasets contain domain-specific jargon or structures absent from pre-training data.
  • Privacy audits must be performed after generation to verify that claimed DP bounds still hold.
  • Existing evaluations without pre-training controls systematically overestimate both utility and privacy of synthetic text.
  • The public pre-training and private generation workflow cannot be trusted to deliver the stated privacy protections on real datasets.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same pre-training leakage pattern could undermine DP guarantees in synthetic data generation for images or structured records.
  • Methods that enforce DP during the entire pre-training stage rather than only at generation time would be needed to restore reliable bounds.

Load-bearing premise

The nine curated datasets and the new membership inference attack sufficiently represent real-world private text distributions and that the observed quality drops and privacy violations extend to other model sizes and DP mechanisms.

What would settle it

A DP text generator that achieves high fidelity and passes the membership inference attack even after the underlying LLM was pre-trained on portions of the target private data would falsify the claim.

Figures

Figures reproduced from arXiv: 2509.14594 by Anil Anthony Bharath, Goran Nenadic, Hao Li, Iqra Zahid, Jie Zhang, Siew Kei Lam, Srinivasan Nandakumar, Viktor Schlegel, Warren Del-Pinto, Yidan Sun, Yulong Wu, Yuping Wu.

Figure 1
Figure 1. Figure 1: Correlation of leakage with averaged (left): Utility, i.e., relative F1 improvement over baseline, (Right): Fidelity, i.e., MAUVE. Spearman correlation at ρ = 0.3 at p ≪ 0.05: and ρ = 0.2 at p = 0.1, respectively. Training Data Leakage As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The ROC curves illustrate the performance of membership inference attacks (MIA) on synthetic data generated [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Left: ϵ-averaged MAUVE scores of datasets generated by DP-SGD trained models of increasing sizes. Right: Advantages of LoRA vs full fine-tuning across different datasets and ϵ, sorted by increasing pre-training leakage. 6 Conclusions This work highlights the persistent challenges in generating high-quality, domain-specific synthetic data with differential privacy. Through a standardized evaluation framewor… view at source ↗
Figure 4
Figure 4. Figure 4: Metric trends across epsilon levels for AUG-PE and DP-Gen Methods. Each subplot shows how a specific [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
read the original abstract

Synthetic text generation with Differential Privacy (DP) guarantees emerges as a principled approach that can enable the sharing of sensitive datasets across institutional and regulatory boundaries, while bounding the risks of re-identification and membership inference. LLM-based methods deliver promising results; however, comparisons are exacerbated by differing evaluation setups and "private" datasets, potential pre-training contamination is not considered and guarantees are not verified with DP audits. To advance this field, we introduce a unified evaluation framework with standardised utility and fidelity metrics and privacy audits, encompassing nine curated datasets that capture domain-specific complexities such as technical jargon, long-context dependencies, and specialised document structures. In a large-scale empirical study, we benchmark LLM-based state-of-the-art DP text generators of varying sizes (between 1--8B). Our results indicate that DP synthetic text generation remains an unsolved challenge, with quality deteriorating more as the private datasets deviate further from the generators' pre-training corpora. Our novel synthetic text membership inference attack (MIA) explains this observation: Synthetic data quality is overestimated when LLMs have been pre-trained -- without DP -- on portions of the "private" data to be generated. Finally, our work provides the first quantitative evidence that this "public pre-training and private generation" paradigm invalidates the guaranteed privacy bounds of real-world private datasets.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces SynBench, a unified benchmark and evaluation framework for differentially private (DP) synthetic text generation with LLMs. It curates nine datasets spanning technical, long-context, and specialized domains; standardizes utility, fidelity, and privacy-audit metrics; and conducts a large-scale study of 1--8B parameter DP generators. Key findings are that synthetic quality degrades as private data deviates from pre-training distributions and that a novel synthetic-text membership inference attack (MIA) demonstrates quality overestimation precisely when pre-training has seen portions of the target data, leading to the claim that the public-pre-training-plus-private-DP-generation paradigm invalidates real-world privacy bounds.

Significance. If the central claims are substantiated, the work is significant: it supplies the first quantitative evidence that pre-training contamination can nullify intended DP guarantees in practical text-generation settings, supplies a reproducible benchmark with standardized audits, and demonstrates a scalable MIA tailored to synthetic text. These contributions could shift evaluation standards and motivate DP methods that explicitly handle distributional mismatch with public pre-training corpora.

major comments (2)
  1. [§4] §4 (MIA and overlap analysis): The novel MIA reports elevated attack AUC when pre-training corpora overlap with the nine 'private' datasets, yet the manuscript provides neither n-gram overlap statistics, decontamination checks, nor an ablation that removes overlapping samples before running the attack. Without these controls the observed success could arise from general distributional or stylistic similarity rather than direct pre-training contamination, which is load-bearing for the causal claim that the paradigm invalidates DP privacy bounds.
  2. [§3.2 and §5.1] §3.2 and §5.1 (deviation metric): The central observation that 'quality deteriorating more as the private datasets deviate further from the generators' pre-training corpora' is presented without an explicit, quantitative measure of deviation (e.g., perplexity of the private data under the base model or embedding-space distance). The correlation therefore remains qualitative and cannot yet support the stronger causal interpretation offered in the abstract and conclusion.
minor comments (2)
  1. [Abstract] Abstract: The phrase 'guarantees are not verified with DP audits' is used to motivate the work; the paper itself performs audits, so the wording should be updated to reflect that prior literature lacked such verification.
  2. [Tables and Figures] Table captions and Figure legends: Ensure every table and figure explicitly states the exact DP parameters (ε, δ) and the base-model pre-training cutoff date used for each row/curve.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment below and commit to revisions that will strengthen the empirical support for our claims without altering the core contributions.

read point-by-point responses

  1. Referee: [§4] §4 (MIA and overlap analysis): The novel MIA reports elevated attack AUC when pre-training corpora overlap with the nine 'private' datasets, yet the manuscript provides neither n-gram overlap statistics, decontamination checks, nor an ablation that removes overlapping samples before running the attack. Without these controls the observed success could arise from general distributional or stylistic similarity rather than direct pre-training contamination, which is load-bearing for the causal claim that the paradigm invalidates DP privacy bounds.

    Authors: We agree that explicit controls are needed to isolate the effect of direct pre-training contamination. The current manuscript does not report n-gram overlap statistics, decontamination procedures, or the requested ablation. In the revised version we will add (i) n-gram overlap statistics between each private dataset and the relevant pre-training corpora, (ii) a decontamination check, and (iii) an ablation that removes overlapping samples before re-running the MIA. These additions will allow readers to assess whether the elevated AUC is attributable to direct overlap or to broader distributional similarity. revision: yes

  2. Referee: [§3.2 and §5.1] §3.2 and §5.1 (deviation metric): The central observation that 'quality deteriorating more as the private datasets deviate further from the generators' pre-training corpora' is presented without an explicit, quantitative measure of deviation (e.g., perplexity of the private data under the base model or embedding-space distance). The correlation therefore remains qualitative and cannot yet support the stronger causal interpretation offered in the abstract and conclusion.

    Authors: We acknowledge that the deviation analysis is currently qualitative. The manuscript relies on domain descriptions rather than a numeric distance metric. In the revision we will introduce a quantitative deviation measure—perplexity of each private dataset under the corresponding base (non-DP) model—and report its correlation with the observed utility and fidelity degradation across the nine datasets. This will be added to §3.2 and §5.1 and will support a more precise statement of the relationship. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical benchmark and MIA results are independent of final claims

full rationale

The paper's derivation chain consists of curating nine datasets, defining standardized utility/fidelity metrics and privacy audits, benchmarking 1-8B DP generators, and introducing a novel synthetic-text MIA whose success rates are measured directly on generated outputs. The central claim—that public pre-training plus private DP generation invalidates real-world privacy bounds—follows from the observed MIA AUCs and quality-overestimation patterns rather than from any self-referential definition, fitted parameter renamed as prediction, or load-bearing self-citation. No equations or ansatzes reduce the result to its inputs by construction; the MIA is presented as an independent diagnostic tool whose outcomes provide the quantitative evidence. The work is therefore self-contained against its own experimental benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The work relies on standard definitions of differential privacy and membership inference; no new free parameters or invented entities are introduced. The main assumptions are that the chosen datasets capture relevant domain complexities and that the MIA faithfully measures privacy leakage.

axioms (2)
  • standard math Standard differential privacy definitions and composition theorems hold for the text generation mechanisms tested.
    Invoked when claiming that the generators provide DP guarantees that are then audited.
  • domain assumption The nine curated datasets are representative of real-world private text with technical jargon and long-context dependencies.
    Central to the claim that quality deterioration generalizes.

pith-pipeline@v0.9.0 · 5801 in / 1442 out tokens · 34733 ms · 2026-05-18T16:42:17.887483+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

49 extracted references · 49 canonical work pages · 6 internal anchors

  1. [1]

    Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang

    Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep Learning with Differential Privacy. InProceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, volume 24-28-October-2016, pages 308–318, New York, NY , USA, 10 2016. ACM. ISBN 9781450341394. doi: 10.1145/2976749.2978318....

    work page doi:10.1145/2976749.2978318 2016

  2. [2]

    Automatic semantic classification of scientific literature according to the hallmarks of cancer.Bioinform., 32(3):432–440,

    Simon Baker, Ilona Silins, Yufan Guo, Imran Ali, Johan H"ogberg, Ulla Stenius, and Anna Korhonen. Automatic semantic classification of scientific literature according to the hallmarks of cancer.Bioinform., 32(3):432–440,

    work page

  3. [3]

    URLhttps://doi.org/10.1093/bioinformatics/btv585

    doi: 10.1093/bioinformatics/btv585. URLhttps://doi.org/10.1093/bioinformatics/btv585. 9 APREPRINT- SEPTEMBER19, 2025

    work page doi:10.1093/bioinformatics/btv585 2025

  4. [4]

    In: International Joint Con- ference on Natural Language Processing (2023)

    Claire Barale, Michael Rovatsos, and Nehal Bhuta. Automated refugee case analysis: A NLP pipeline for supporting legal practitioners. InFindings of the Association for Computational Linguistics: ACL 2023, pages 2992–3005, Toronto, Canada, jul 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023. findings-acl.187. URLhttps://aclanthology....

    work page doi:10.18653/v1/2023 2023

  5. [5]

    Lof: identifying density-based local outliers

    Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. Lof: identifying density-based local outliers. InProceedings of the 2000 ACM SIGMOD international conference on Management of data, pages 93–104, 2000

    work page 2000

  6. [6]

    The secret sharer: Evaluating and testing unintended memorization in neural networks

    Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In28th USENIX security symposium (USENIX security 19), pages 267–284, 2019

    work page 2019

  7. [7]

    Extracting Training Data from Large Language Models

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-V oss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel. Extracting Training Data from Large Language Models. In30th USENIX Security Symposium (USENIX Security 21), pages 2633–2650,

    work page

  8. [8]

    URLhttp://arxiv.org/abs/2012.07805

    work page arXiv 2012

  9. [9]

    Membership Inference Attacks From First Principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. Membership Inference Attacks From First Principles. In2022 IEEE Symposium on Security and Privacy (SP), pages 1897–1914. IEEE, 5 2022. ISBN 978-1-6654-1316-9. doi: 10.1109/SP46214.2022.9833649. URL https://ieeexplore. ieee.org/document/9833649/

    work page doi:10.1109/sp46214.2022.9833649 1914

  10. [10]

    Large-Scale Multi-Label Text Classification on EU Legislation

    Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, and Ion Androutsopoulos. Large-scale multi-label text classification on eu legislation.arXiv preprint arXiv:1906.02192, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1906

  11. [11]

    Mollick, Hila Lifshitz-Assaf, Katherine Kellogg, Saran Rajendran, Lisa Krayer, François Candelon, and Karim R

    Fabrizio Dell’Acqua, Edward McFowland, Ethan R. Mollick, Hila Lifshitz-Assaf, Katherine Kellogg, Saran Rajendran, Lisa Krayer, François Candelon, and Karim R. Lakhani. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality.SSRN Electronic Journal, 2023. ISSN 1556-5068. do...

    work page doi:10.2139/ssrn.4573321 2023

  12. [12]

    Pierrehumbert, and Stefan Zohren

    Felix Drinkall, Janet B. Pierrehumbert, and Stefan Zohren. When dimensionality hurts: The role of llm embedding compression for noisy regression tasks, 2025

    work page 2025

  13. [13]

    The llama 3 herd of models.arXiv e-prints, pages arXiv–2407, 2024

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv e-prints, pages arXiv–2407, 2024

    work page 2024

  14. [14]

    Differential privacy

    Cynthia Dwork. Differential privacy. In Michele Bugliesi, Bart Preneel, Vladimiro Sassone, and Ingo Wegener, ed- itors,Automata, Languages and Programming, pages 1–12, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978-3-540-35908-1

    work page 2006

  15. [15]

    The algorithmic foundations of differential privacy.Foundations and Trends® in Theoretical Computer Science, 9(3–4):211–407, 2014

    Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy.Foundations and Trends® in Theoretical Computer Science, 9(3–4):211–407, 2014

    work page 2014

  16. [16]

    Differential privacy in practice: Expose your epsilons! Journal of Privacy and Confidentiality, 9(2), 2019

    Cynthia Dwork, Nitin Kohli, and Deirdre Mulligan. Differential privacy in practice: Expose your epsilons! Journal of Privacy and Confidentiality, 9(2), 2019

    work page 2019

  17. [17]

    Differentially Private Knowledge Distillation via Synthetic Text Generation

    James Flemings and Murali Annavaram. Differentially Private Knowledge Distillation via Synthetic Text Generation. InFindings of the Association for Computational Linguistics ACL 2024, pages 12957–12968, Stroudsburg, PA, USA, 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-acl.769. URLhttps://aclanthology.org/2024.findings-acl.769

    work page doi:10.18653/v1/2024.findings-acl.769 2024

  18. [18]

    Differentially Private Next-Token Prediction of Large Language Models

    James Flemings, Meisam Razaviyayn, and Murali Annavaram. Differentially Private Next-Token Prediction of Large Language Models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4390–4404, Stroudsburg, PA, USA, 3 2024. Association ...

    work page doi:10.18653/v1/2024.naacl-long.247 2024

  19. [19]

    The elusive pursuit of reproducing pate-gan: Benchmarking, auditing, debugging.arXiv preprint arXiv:2406.13985, 2024

    Georgi Ganev, Meenatchi Sundaram Muthu Selva Annamalai, and Emiliano De Cristofaro. The elusive pursuit of reproducing pate-gan: Benchmarking, auditing, debugging.arXiv preprint arXiv:2406.13985, 2024

    work page arXiv 2024

  20. [20]

    When differential privacy meets NLP: The devil is in the detail

    Ivan Habernal. When differential privacy meets NLP: The devil is in the detail. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1522–1528, Stroudsburg, PA, USA,

    work page 2021

  21. [21]

    doi: 10.18653/v1/2021.emnlp-main.114

    Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.114

    work page doi:10.18653/v1/2021.emnlp-main.114 2021

  22. [22]

    Snomed ct entity linking challenge (version 1.0.0)

    Will Hardman, Mark Banks, Rory Davidson, Donna Truran, Nindya Widita Ayuningtyas, Hoa Ngo, Alistair Johnson, and Tom Pollard. Snomed ct entity linking challenge (version 1.0.0). physionet. rrid:scr_007345. https://doi.org/10.13026/s48e-sp45, 2023. 10 APREPRINT- SEPTEMBER19, 2025

    work page doi:10.13026/s48e-sp45 2023

  23. [23]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Information Systems, 43(2):1–55, 2025

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Information Systems, 43(2):1–55, 2025

    work page 2025

  24. [24]

    Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J

    Alistair E.W. Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J. Pollard, Benjamin Moody, Brian Gow, Li wei H. Lehman, Leo A. Celi, and Roger G. Mark. MIMIC-IV , a freely accessible electronic health record dataset.Scientific Data, 10(1), 12 2023. ISSN 20524463. doi: 10.1038/ S41597-022-01899-X

    work page 2023

  25. [25]

    The Composition Theorem for Differential Privacy, 6 2015

    Peter Kairouz, Sewoong Oh, and Pramod Viswanath. The Composition Theorem for Differential Privacy, 6 2015. ISSN 1938-7228. URLhttps://proceedings.mlr.press/v37/kairouz15.html

    work page 2015

  26. [26]

    Scaling Laws for Neural Language Models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2001

  27. [27]

    Privacy-preserving retrieval-augmented generation with differential privacy.arXiv preprint arXiv:2412.04697, 2024

    Tatsuki Koga, Ruihan Wu, and Kamalika Chaudhuri. Privacy-preserving retrieval-augmented generation with differential privacy.arXiv preprint arXiv:2412.04697, 2024

    work page arXiv 2024

  28. [28]

    How much is enough? choosing ε for differential privacy

    Jaewoo Lee and Chris Clifton. How much is enough? choosing ε for differential privacy. InInternational Conference on Information Security, pages 325–340. Springer, 2011

    work page 2011

  29. [29]

    Team:PULSAR at ProbSum 2023:PULSAR: Pre-training with Extracted Healthcare Terms for Summarising Patients’ Problems and Data Augmentation with Black-box Large Language Models

    Hao Li, Yuping Wu, Viktor Schlegel, Riza Batista-Navarro, Thanh-Tung Nguyen, Abhinav Ramesh Kashyap, Xiao-Jun Zeng, Daniel Beck, Stefan Winkler, and Goran Nenadic. Team:PULSAR at ProbSum 2023:PULSAR: Pre-training with Extracted Healthcare Terms for Summarising Patients’ Problems and Data Augmentation with Black-box Large Language Models. InThe 22nd Worksh...

    work page doi:10.18653/v1/2023.bionlp-1.49 2023

  30. [30]

    Differentially Private Language Models for Secure Data Sharing

    Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schoelkopf, and Mrinmaya Sachan. Differentially Private Language Models for Secure Data Sharing. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4860–4873, Stroudsburg, PA, USA, 10 2022. Association for Computational Linguistics. doi: 10.18653/v1/202...

    work page doi:10.18653/v1/2022.emnlp-main.323 2022

  31. [31]

    The canary’s echo: Auditing privacy risks of llm-generated synthetic text.arXiv preprint arXiv:2502.14921, 2025

    Matthieu Meeus, Lukas Wutschitz, Santiago Zanella-Béguelin, Shruti Tople, and Reza Shokri. The canary’s echo: Auditing privacy risks of llm-generated synthetic text.arXiv preprint arXiv:2502.14921, 2025

    work page arXiv 2025

  32. [32]

    Scaling data-constrained language models.Advances in Neural Information Processing Systems, 36:50358–50376, 2023

    Niklas Muennighoff, Alexander Rush, Boaz Barak, Teven Le Scao, Nouamane Tazi, Aleksandra Piktus, Sampo Pyysalo, Thomas Wolf, and Colin A Raffel. Scaling data-constrained language models.Advances in Neural Information Processing Systems, 36:50358–50376, 2023

    work page 2023

  33. [33]

    Differential privacy: Future work & open challenges

    Joseph Near and David Darais. Differential privacy: Future work & open challenges. https://www.nist.gov/ blogs/cybersecurity-insights/differential-privacy-future-work-open-challenges , 2022. NIST Cybersecurity Insights Blog, January 24, 2022

    work page 2022

  34. [34]

    GPT-4 Technical Report

    OpenAI. GPT-4 Technical Report.arXiv:2303.08774, 3 2023. URL https://arxiv.org/abs/2303.08774v6

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  35. [35]

    Scalable Private Learning with PATE

    Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Ulfar Erlingsson. Scalable Private Learning with PATE. InInternational Conference on Learning Representations, 2 2018. URL http://arxiv.org/abs/1802.08908

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  36. [36]

    MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers

    Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, Zaid Harchaoui, and Paul G Allen. MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers. InAdvances in Neural Information Processing Systems, volume 34, pages 4816–4828, 12 2021. URL https://github.com/krishnap25/mauve

    work page 2021

  37. [37]

    Generating Datasets with Pretrained Language Models

    Timo Schick and Hinrich Schütze. Generating Datasets with Pretrained Language Models. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6943–6951, Stroudsburg, PA, USA, 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.555

    work page doi:10.18653/v1/2021.emnlp-main.555 2021

  38. [38]

    Generating synthetic data with formal privacy guarantees: State of the art and the road ahead.arXiv preprint arXiv:2503.20846, 2025

    Viktor Schlegel, Anil A Bharath, Zilong Zhao, and Kevin Yee. Generating synthetic data with formal privacy guarantees: State of the art and the road ahead.arXiv preprint arXiv:2503.20846, 2025

    work page arXiv 2025

  39. [39]

    Detecting Pretraining Data from Large Language Models

    Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models.arXiv preprint arXiv:2310.16789, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  40. [40]

    Evaluating differentially private generation of domain-specific text.arXiv preprint arXiv:2508.20452, 2025

    Yidan Sun, Viktor Schlegel, Srinivasan Nandakumar, Iqra Zahid, Yuping Wu, Warren Del-Pinto, Goran Nenadic, Siew-Kei Lam, Jie Zhang, and Anil A Bharath. Evaluating differentially private generation of domain-specific text.arXiv preprint arXiv:2508.20452, 2025. 11 APREPRINT- SEPTEMBER19, 2025

    work page arXiv 2025

  41. [41]

    Inan, Andre Manoel, Fatemehsadat Mireshghallah, Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, and Robert Sim

    Xinyu Tang, Richard Shin, Huseyin A. Inan, Andre Manoel, Fatemehsadat Mireshghallah, Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, and Robert Sim. Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation. 9 2023. URLhttp://arxiv.org/abs/2309.11765

    work page arXiv 2023

  42. [42]

    Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2000

    Virginia Teller. Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2000

    work page 2000

  43. [43]

    Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining, 7 2024

    Florian Tramèr, Gautam Kamath, and Nicholas Carlini. Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining, 7 2024. ISSN 2640-3498. URL https://proceedings.mlr. press/v235/tramer24a.html

    work page 2024

  44. [44]

    Recognizing obesity and comorbidities in sparse data.Journal of the American Medical Informatics Association, 16(4):561–570, 07 2009

    Ozlem Uzuner. Recognizing obesity and comorbidities in sparse data.Journal of the American Medical Informatics Association, 16(4):561–570, 07 2009. doi: 10.1197/jamia.M3115. URL https://doi.org/10.1197/jamia. M3115

    work page doi:10.1197/jamia.m3115 2009

  45. [45]

    BloombergGPT: A Large Language Model for Finance

    Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance.arXiv preprint arXiv:2303.17564, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  46. [46]

    Differentially Private Synthetic Data via Foundation Model APIs 2: Text

    Chulin Xie, Zinan Lin, Arturs Backurs, Sivakanth Gopi, Da Yu, Huseyin A Inan, Harsha Nori, Haotian Jiang, Huishuai Zhang, Yin Tat Lee, Bo Li, and Sergey Yekhanin. Differentially Private Synthetic Data via Foundation Model APIs 2: Text. InProceedings of the 41st International Conference on Machine Learning, pages 54531– 54560. PMLR, 3 2024. URLhttp://arxiv...

    work page arXiv 2024

  47. [47]

    Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim

    Xiang Yue, Huseyin A. Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim. Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe.Proceedings of the Annual Meeting of the Association for Computational Linguistics, 1:1321–1342, 2023. ISSN 0736587X. doi: 10.18653/V1/2023.ACL-LONG...

    work page doi:10.18653/v1/2023.acl-long.74 2023

  48. [48]

    Siren’s song in the ai ocean: A survey on hallucination in large language models.Computational Linguistics, pages 1–45, 2025

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren’s song in the ai ocean: A survey on hallucination in large language models.Computational Linguistics, pages 1–45, 2025

    work page 2025

  49. [49]

    sentence- t5-base

    Maryam Zolnoori, Kin Wah Fung, Timothy B. Patrick, Paul Fontelo, Hadi Kharrazi, Anthony Faiola, Yi Shuan Shirley Wu, Christina E. Eldredge, Jake Luo, Mike Conway, Jiaxi Zhu, Soo Kyung Park, Kelly Xu, Hamideh Moayyed, and Somaieh Goudarzvand. A systematic approach for developing a corpus of patient reported adverse drug events: A case study for SSRI and SN...

    work page doi:10.1016/j.jbi.2018.12.005 2019