Pith. sign in

REVIEW 4 major objections 4 minor 41 references

Stronger differential-privacy guarantees make LLM-generated text shorter, less grammatical, and less diverse, and the deterioration transfers to classifiers trained on it.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 16:57 UTC pith:AVXNZTNJ

load-bearing objection Solid empirical sweep on DP's language-quality costs; headline percentages miscalculated, but the directional result stands. the 4 major comments →

arxiv 2509.11176 v1 pith:AVXNZTNJ submitted 2025-09-14 cs.CL cs.AI

Differentially-private text generation degrades output language quality

Erion \c{C}ano , Ivan Habernal This is my paper
classification cs.CL cs.AI
keywords differential privacyLLM fine-tuningsynthetic text generationtext qualitylexical diversitygrammaticalitydata utilitydownstream classification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that tuning LLMs under differential privacy systematically damages the language of the texts they generate. Across five models and three corpora, stronger privacy (smaller epsilon) yields outputs that are shorter, less grammatically correct, and less lexically diverse, with length dropping by at least 77% as measured, grammaticality falling by at least 9%, and bigram diversity falling by at least 10%. The same trend appears in downstream classifiers trained on the synthetic output: accuracy drops, and the effect is severe on a difficult 227-class genre task. The paper frames this as a previously unmeasured cost of private synthetic data generation, with the explicit takeaway that adopting DP-tuning for synthetic text requires a careful analysis of the privacy-quality tradeoff.

Core claim

Under (ε,δ)-differential privacy, the noise injected during fine-tuning imposes a monotonic quality penalty: as ε decreases from 10 to 5 to 1, average output length, grammatical correctness, and n-gram diversity all decline, while compression ratio (an inverse diversity signal) rises. The authors report length reductions by factors between 1.77 and 5.94, grammaticality regressions between 1.09 and 1.67, and bigram-diversity regressions between 1.1 and 1.4 across model-dataset combinations. Classifiers (ModernBERT) tuned on this synthetic text lose accuracy—mildly on a 44-class cause-of-death task, dramatically on a 227-class book-genre task, with F1 regression factors up to 15. The non-priva

What carries the argument

The privacy budget ε in (ε,δ)-DP fine-tuning, which controls the scale of noise added to gradients. The paper compares outputs at ε = 1, 5, 10, and ∞ (no DP) and tracks language quality with seven metrics: length (in characters and tokens), word-level and sentence-level grammaticality (1 − errors/tokens), distinct unigram and bigram diversity, homogenization score, and gzip compression ratio.

Load-bearing premise

The automatic metrics and uncontrolled decoding/classification settings are assumed to isolate DP as the cause—if outputs are truncated by a token cap or classifiers are trained on unequal sample sizes, the measured degradation could be partly an artifact.

What would settle it

Generate synthetic texts from ε = 1 and ε = ∞ models with a fixed token budget (e.g., always decode to 300 tokens), then re-measure grammaticality and diversity; if the ε = 1 outputs no longer show the 9% and 10% drops, the length collapse causes the quality loss. Alternatively, human annotators rating randomly ordered ε = ∞ and ε = 1 samples for fluency would test whether the automatic grammar metric reflects perceived quality.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Stronger privacy guarantees (smaller ε) produce shorter, less grammatical, and less diverse synthetic texts across all five tested LLMs and all three corpora.
  • The quality penalty scales monotonically with privacy strength: quality declines steadily from ε = ∞ to ε = 1, so the choice of ε directly budgets the linguistic cost.
  • Downstream classifiers trained on DP-synthetic data lose accuracy, with the effect amplified on harder, many-class tasks such as distinguishing 227 book genres.
  • Since non-private synthetic baselines closely match natural texts in the measured metrics, the degradation is attributable to DP itself, not to synthetic generation per se.
  • Privacy-preserving synthetic data pipelines must account for these quality losses when selecting ε, especially for high-cardinality or information-dense tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The length collapse may itself drive the grammaticality and diversity drops, since shorter texts naturally reduce n-gram diversity and per-token error counts; a fixed-length decoding experiment would clarify whether DP damages language directly or mainly truncates outputs.
  • The 227-class genre result suggests the utility cost of DP scales with task difficulty, so practitioners with large label spaces may need substantially larger privacy budgets or alternative safeguards.
  • Because the metrics are automatic and reference-free, human evaluation or a reference-based fluency measure could verify whether the measured declines are perceptible to real users of synthetic data.
  • A testable extension: DP-specific decoding strategies (longer generation, repetition penalties, or constrained decoding) might recover part of the lost quality without weakening the privacy guarantee.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper reports an empirical study of how differentially private (DP) fine-tuning affects the language quality and downstream utility of synthetic text generated by LLMs. Five open LLMs (Bloom-7B, Phi-4-mini, Phi-4-medium, Qwen-2.5-7B, Qwen-2.5-14B) are fine-tuned on three corpora (verbal autopsies, Reddit suicide-related posts, book summaries) under four privacy levels (ε = ∞, 10, 5, 1). The authors measure output length, grammatical correctness, lexical diversity, and performance of a ModernBERT classifier trained on the synthetic texts. The claimed findings are that stronger privacy constraints produce texts that are shorter by at least 77%, less grammatically correct by at least 9%, and less diverse by at least 10% in bigram diversity, with corresponding drops in downstream classification accuracy.

Significance. The question addressed is timely and practically important: if DP fine-tuning systematically degrades synthetic text quality, this is a significant cost that practitioners should weigh against privacy benefits. The study's breadth is a strength: it covers multiple models, multiple datasets, several quality metrics, and downstream utility, and it compares DP outputs against both natural texts and non-private synthetic baselines. The direction of the findings is consistent across the appendix tables. However, the headline quantitative claims are undermined by arithmetic errors, and the experimental sections omit several details that are necessary to attribute the measured differences to DP rather than to uncontrolled generation or classification settings. The core conclusion is credible, but the paper needs substantial revision before the specific claims can be accepted.

major comments (4)
  1. [Abstract, §4.1, §5] The claim that outputs are 'shorter by at least 77%' is contradicted by the paper's own tables. A regression factor of 1.77 corresponds to a 1 − 1/1.77 ≈ 43.5% decrease, and a factor of 5.94 corresponds to a 83.2% decrease, not 494% (a decrease cannot exceed 100%). Recomputing from Tables 11–25 gives a minimum NT decrease from ε=∞ to ε=1 of about 43.4% (Bloom7b, Suicide: 145→82) and a maximum of about 83.2% (Bloom7b, Autopsy: 184→31). Thus the abstract's 'at least 77%' lower bound is false, and the Discussion's 'at most 494%' is meaningless. These percentages must be corrected throughout.
  2. [§4.2, §4.3, Abstract] The 'at least 9%' grammaticality loss and 'at least 10%' bigram-diversity loss are likewise not supported by the reported factors. A factor of 1.09 is an 8.3% decrease, and a factor of 1.10 is a 9.1% decrease, both below the claimed lower bounds. If the intended claim is 'approximately 9–10%' or a range computed from the actual score differences, it should be stated precisely. As written, the 'at least' phrasing is arithmetically inconsistent with the figures in §4.2 and §4.3.
  3. [§4.4, Tables 26–35] The classification regression factors reported in §4.4 do not match the appendix tables. For example, for Booksum classification the text reports accuracy regression factors of 3.89, 4, and 3.44, but Table 27 gives 0.43/0.185 ≈ 2.32 for PhiMed, Table 31 gives 0.356/0.097 ≈ 3.67 for Bloom7b, and Table 35 gives 0.316/0.096 ≈ 3.29 for Qwen14b. The F1 factors (13.5, 15, 7.33) similarly disagree with the tables (e.g., Table 27: 0.331/0.096 ≈ 3.45). The text and tables must be reconciled; this is load-bearing because the classification utility decline is a central claim.
  4. [§3.3, §A, Appendix C] The grammaticality metric in Eq. (1) requires an error detector, but the detector is never named or referenced. This makes the grammaticality results impossible to reproduce and leaves open the possibility that the detector's behavior on short, DP-generated texts drives the reported decline. Also missing are the decoding parameters used for generation (temperature, top-p, max tokens, number of samples) and the classification training details (number of synthetic samples per ε, epochs, batch size, learning rate, compute). Without these, the causal attribution to DP is not fully isolated; for instance, if classifiers were trained on unequal numbers of samples, the accuracy comparisons are biased. These details must be supplied.
minor comments (4)
  1. [Throughout] Typographical issues: 'constrains' should be 'constraints' (Abstract, §1, §5); 'Authopsy' in §4.4 should be 'Autopsy'; 'ModerBERT' in §4.4 should be 'ModernBERT'; 'the later' in §2.1 should be 'the latter'; 'do now reveal' in §7 should be 'do not reveal'.
  2. [§4.1, §5] The term 'regression factor' is used for a simple ratio of mean lengths or scores; this is not a regression coefficient and may confuse readers. Consider using 'ratio' or 'fold-change' consistently.
  3. [Figures 1–3] The figures show point estimates without error bars or confidence intervals. Given that the paper makes 'at least' claims, some measure of variability (e.g., bootstrap intervals or standard deviations across samples) would strengthen the quantitative statements.
  4. [§2.2] The description of the grammaticality metric would benefit from a citation to the specific GBM implementation used. Also, the phrase 'errors/tokens' should specify whether tokens are whitespace-split words or subwords.

Circularity Check

0 steps flagged

No circular derivation found; the headline '77%' is an arithmetic mislabeling, not a circular step.

full rationale

This paper is an empirical measurement study, not a derivation from first principles. The central quantities—output length (NT), grammatical correctness (SG/WG), lexical diversity (BD/CR), and downstream classification accuracy—are all direct metrics computed from generated texts or from fine-tuned classifiers. No parameter is fitted to a subset of the data and then reported as a prediction: the 'regression factors' are simply ratios of observed mean scores between the ε=∞ and ε=1 conditions (e.g., NT 184/31 for Bloom7b on Autopsy), and the percentages in the abstract are the authors' (incorrect) conversion of these ratios. That conversion error is a correctness problem, not circularity: the underlying measurements are independent of the claims. The only self-citation, Ochs and Habernal (2024), appears as background in Section 2.1 and is not load-bearing; no result is justified by it. There is no self-definitional reduction, no fitted input renamed as a prediction, no uniqueness argument imported from the authors' prior work, and no ansatz smuggled in via citation. The paper's claims, while potentially vulnerable to concerns about uncontrolled generation parameters or unequal classifier training data, are not equivalent to their inputs by construction. Therefore, per the circularity criteria, the appropriate score is 0.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

No numbers are fitted to make the central claim; all reported quantities are measured scores, and no new theoretical entities are postulated. The claim rests on methodological assumptions about metric validity, constant decoding settings, correct DP implementation, and comparable classifier training. None of these are fully documented.

axioms (4)
  • domain assumption The grammatical correctness metric GC = 1 - errors/tokens (Equation 1) is a valid proxy for language quality, and the unnamed error-detection tool counts errors reliably on short degenerate outputs.
    Section 3.3 invokes grammaticality-based metrics from Napoles et al. (2016) but never identifies the error detector used to count #errors. The central grammaticality result depends on this tool.
  • domain assumption Generation decoding settings (temperature, max tokens, top-p, prompts) are held constant across privacy levels, so length and diversity differences are caused by DP.
    Section 3.1 and Appendix A report only fine-tuning hyperparameters. No decoding parameters are given; if max-new-tokens differed by configuration, the length conclusion would be confounded.
  • domain assumption The DP-SGD implementation and privacy accounting used to realize the stated (epsilon, delta) values are correct.
    Section 3.1 gives epsilon values and a delta formula delta = 1/(n*10), but no DP library, accountant, or implementation details are cited, so privacy guarantees are taken on faith.
  • domain assumption Fine-tuning ModernBERT on the synthetic data is a valid downstream-utility probe, and training sets are comparable across privacy levels.
    Section 4.4 reports classification accuracy and F1 but does not state whether the number of synthetic training samples, training epochs, or label distributions were equal across epsilon levels.

pith-pipeline@v1.3.0-alltime-deepseek · 17779 in / 10347 out tokens · 125297 ms · 2026-08-04T16:57:30.686725+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Differentially-private text generation degrades output language quality." pith.science (2026). https://pith.science/paper/AVXNZTNJ

@misc{pith2026250911176,
  author       = {Pith},
  title        = {Pith review of: Differentially-private text generation degrades output language quality},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AVXNZTNJ}},
  note         = {Machine review of arXiv:2509.11176}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Ensuring user privacy by synthesizing data from large language models (LLMs) tuned under differential privacy (DP) has become popular recently. However, the impact of DP fine-tuned LLMs on the quality of the language and the utility of the texts they produce has not been investigated. In this work, we tune five LLMs with three corpora under four levels of privacy and assess the length, the grammatical correctness, and the lexical diversity of the text outputs they produce. We also probe the utility of the synthetic outputs in downstream classification tasks such as book genre recognition based on book descriptions and cause of death recognition based on verbal autopsies. The results indicate that LLMs tuned under stronger privacy constrains produce texts that are shorter by at least 77 %, that are less grammatically correct by at least 9 %, and are less diverse by at least 10 % in bi-gram diversity. Furthermore, the accuracy they reach in downstream classification tasks decreases, which might be detrimental to the usefulness of the generated synthetic data.

Figures

Figures reproduced from arXiv: 2509.11176 by Erion \c{C}ano, Ivan Habernal.

Figure 1
Figure 1. Figure 1: Number of tokens (NT) for each privacy level and dataset (Autopsy on (a), Booksum on (b), and Suicide [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Bigram diversity (BD) for each privacy level and dataset (Autopsy on (a), Booksum on (b), and Suicide [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Classification accuracy and F1 scores for Autopsy on (a) and (b), and for Booksum on (c) and (d). [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 8 canonical work pages · 1 internal anchor

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Hewett, Mojan Javaheripi, Piero Kauffmann, James R

    Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu,...

    Pith/arXiv arXiv 2024

  4. [4]

    Shahad Alkamli and Reham Alabduljabbar. 2024. https://doi.org/10.1016/j.heliyon.2024.e39087 Understanding privacy concerns in chatgpt: A data-driven approach with lda topic modeling . Heliyon, 10(20):e39087

    work page doi:10.1016/j.heliyon.2024.e39087 2024

  5. [5]

    Kareem Amin, Alex Bie, Weiwei Kong, Alexey Kurakin, Natalia Ponomareva, Umar Syed, Andreas Terzis, and Sergei Vassilvitskii. 2024. https://doi.org/10.48550/arXiv.2407.12108 Private prediction for large-scale synthetic text generation . arXiv preprint. Synt-12

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.12108 2024

  6. [6]

    David Bamman and Noah A. Smith. 2013. https://arxiv.org/abs/1305.1319 New alignment methods for discriminative book summarization . CoRR, abs/1305.1319

    Pith/arXiv arXiv 2013

  7. [7]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  8. [8]

    Erion C ano and Ond r ej Bojar. 2019. https://doi.org/10.18653/v1/W19-8630 Efficiency metrics for data-driven models: A text summarization case study . In Proceedings of the 12th International Conference on Natural Language Generation, pages 229--239, Tokyo, Japan. Association for Computational Linguistics

    work page doi:10.18653/v1/w19-8630 2019

  9. [9]

    Daniel Dahlmeier and Hwee Tou Ng. 2012. https://aclanthology.org/N12-1067/ Better evaluation for grammatical error correction . In Proceedings of the 2012 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 568--572, Montr \'e al, Canada. Association for Computational Linguistics

    2012

  10. [10]

    Cynthia Dwork. 2006. Differential privacy. In Automata, Languages and Programming, pages 1--12, Berlin, Heidelberg. Springer Berlin Heidelberg

    2006

  11. [11]

    Cynthia Dwork and Aaron Roth. 2014. https://doi.org/10.1561/0400000042 The algorithmic foundations of differential privacy . Found. Trends Theor. Comput. Sci., 9(3–4):211–407

    doi:10.1561/0400000042 2014

  12. [12]

    Flaxman, Lisa Harman, Jonathan Joseph, Jonathan Brown, and Christopher J.L

    Abraham D. Flaxman, Lisa Harman, Jonathan Joseph, Jonathan Brown, and Christopher J.L. Murray. 2018. https://doi.org/10.12688/gatesopenres.12812.1 A de-identified database of 11,979 verbal autopsy open-ended responses . Gates Open Research, 2:18. Data-1

    work page doi:10.12688/gatesopenres.12812.1 2018

  13. [13]

    Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. 2023. http://arxiv.org/abs/2210.08933 DiffuSeq : Sequence to Sequence Text Generation with Diffusion Models . arXiv preprint. ArXiv:2210.08933 [cs]

    Pith/arXiv arXiv 2023

  14. [15]

    Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, and Chloé Clavel. 2024 b . https://doi.org/10.18653/v1/2024.findings-naacl.228 The Curious Decline of Linguistic Diversity : Training Language Models on Synthetic Text . In Findings of the Association for Computational Linguistics : NAACL 2024 , pages 3589--3604, Mexico City, Mexico. Association for Computa...

    doi:10.18653/v1/2024.findings-naacl.228 2024

  15. [16]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . CoRR, abs/2106.09685

    Pith/arXiv arXiv 2021

  16. [17]

    Khairul Islam, Andrew Wang, Tianhao Wang, Yangfeng Ji, Judy Fox, and Jieyu Zhao

    Md. Khairul Islam, Andrew Wang, Tianhao Wang, Yangfeng Ji, Judy Fox, and Jieyu Zhao. 2024. https://arxiv.org/abs/2410.18749 Does differential privacy impact bias in pretrained nlp models? Preprint, arXiv:2410.18749

    Pith/arXiv arXiv 2024

  17. [18]

    Shaoxiong Ji, Xue Li, Zi Huang, and Erik Cambria. 2022. https://doi.org/10.1007/s00521-021-06208-y Suicidal ideation and mental disorder detection with attentive relation networks . Neural Computing and Applications, 34(13):10309--10319. Data-3

    work page doi:10.1007/s00521-021-06208-y 2022

  18. [19]

    Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. https://doi.org/10.18653/v1/N16-1014 A diversity-promoting objective function for neural conversation models . In Proceedings of the 2016 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 110--119, San ...

    doi:10.18653/v1/n16-1014 2016

  19. [20]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013/ ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  20. [21]

    Xuyang Liu, Zichen Wen, Shaobo Wang, Junjie Chen, Zhishan Tao, Yubo Wang, Xiangqi Jin, Chang Zou, Yiyu Wang, Chenfei Liao, Xu Zheng, Honggang Chen, Weijia Li, Xuming Hu, Conghui He, and Linfeng Zhang. 2025. https://arxiv.org/abs/2505.19147 Shifting ai efficiency from model-centric to data-centric compression . Preprint, arXiv:2505.19147

    arXiv 2025

  21. [22]

    Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schoelkopf, and Mrinmaya Sachan. 2022 a . https://doi.org/10.18653/v1/2022.emnlp-main.323 Differentially Private Language Models for Secure Data Sharing . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 4860--4873, Abu Dhabi, United Arab Emirates. ...

    doi:10.18653/v1/2022.emnlp-main.323 2022

  22. [23]

    Justus Mattern, Benjamin Weggenmann, and Florian Kerschbaum. 2022 b . https://doi.org/10.48550/arXiv.2205.02130 The Limits of Word Level Differential Privacy . arXiv preprint. Priv-51

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2205.02130 2022

  23. [24]

    Microsoft and Authors. 2025. https://arxiv.org/abs/2503.01743 Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras . Preprint, arXiv:2503.01743

    Pith/arXiv arXiv 2025

  24. [25]

    Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2015. https://doi.org/10.3115/v1/P15-2097 Ground Truth for Grammaticality Correction Metrics . In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing ( Volume 2: Short Papers ) , ...

    doi:10.3115/v1/p15-2097 2015

  25. [26]

    Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2016. https://doi.org/10.18653/v1/D16-1228 There's No Comparison : Reference -less Evaluation Metrics in Grammatical Error Correction . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages 2109--2115, Austin, Texas. Association for Computational Linguistics

    work page doi:10.18653/v1/d16-1228 2016

  26. [27]

    Ngong, Joseph Near, and Niloofar Mireshghallah

    Ivoline C. Ngong, Joseph Near, and Niloofar Mireshghallah. 2025. https://doi.org/10.18653/v1/2025.naacl-long.455 Differentially private learning needs better model initialization and self-distillation . In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies...

    work page doi:10.18653/v1/2025.naacl-long.455 2025

  27. [28]

    Sebastian Ochs and Ivan Habernal. 2024. https://arxiv.org/abs/2410.22971 Private synthetic text generation with diffusion models . Preprint, arXiv:2410.22971

    Pith/arXiv arXiv 2024

  28. [29]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics

    arXiv 2002

  29. [30]

    Qwen and Authors. 2025. https://arxiv.org/abs/2412.15115 Qwen2.5 technical report . Preprint, arXiv:2412.15115

    Pith/arXiv arXiv 2025

  30. [31]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf Language models are unsupervised multitask learners . OpenAI. Accessed: 2024-11-15

    2019

  31. [32]

    Siu, Byron C

    Chantal Shaib, Joe Barrow, Jiuding Sun, Alexa F. Siu, Byron C. Wallace, and Ani Nenkova. 2024. https://doi.org/10.48550/ARXIV.2403.00553 Standardizing the Measurement of Text Diversity : A Tool and a Comparative Analysis of Scores . arXiv preprint. Version Number: 1

    doi:10.48550/arxiv.2403.00553 2024

  32. [33]

    Shuang Song, Kamalika Chaudhuri, and Anand D. Sarwate. 2013. https://doi.org/10.1109/GlobalSIP.2013.6736861 Stochastic gradient descent with differentially private updates . In 2013 IEEE Global Conference on Signal and Information Processing, pages 245--248

    arXiv 2013

  33. [34]

    Sanjari Srivastava, Piotr Mardziel, Zhikhun Zhang, Archana Ahlawat, Anupam Datta, and John C Mitchell. 2024. https://arxiv.org/abs/2402.04489 De-amplifying bias from differential privacy in language model fine-tuning . Preprint, arXiv:2402.04489

    Pith/arXiv arXiv 2024

  34. [35]

    Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, and Marius Hobbhahn. 2024. https://arxiv.org/abs/2211.04325 Will we run out of data? Limits of LLM scaling based on human-generated data . Preprint, arXiv:2211.04325

    Pith/arXiv arXiv 2024

  35. [36]

    Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. 2024. https://doi.org/10.48550/arXiv.2412.13663 Smarter, Better , Faster , Longer : A Modern Bidirectional Encoder for Fast , Memory Effi...

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.13663 2024

  36. [37]

    BigScience Workshop . 2022. https://doi.org/10.48550/arXiv.2211.05100 BLOOM: A 176B-Parameter Open-Access Multilingual Language Model . arXiv e-prints, arXiv:2211.05100

    Pith/arXiv arXiv doi:10.48550/arxiv.2211.05100 2022

  37. [38]

    Xiaodong Wu, Ran Duan, and Jianbing Ni. 2024. https://doi.org/10.1016/j.jiixd.2023.10.007 Unveiling security, privacy, and ethical concerns of chatgpt . Journal of Information and Intelligence, 2(2):102--115

    doi:10.1016/j.jiixd.2023.10.007 2024

  38. [39]

    Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, and Huishuai Zhang

    Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A. Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, and Huishuai Zhang. 2022. http://arxiv.org/abs/2110.06500 Differentially Private Fine -tuning of Language Models . arXiv preprint. ArXiv:2110.06500 [cs]

    Pith/arXiv arXiv 2022

  39. [40]

    Xiang Yue, Huseyin Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim. 2023. https://doi.org/10.18653/v1/2023.acl-long.74 Synthetic Text Generation with Differential Privacy : A Simple and Practical Recipe . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics ( Volu...

    doi:10.18653/v1/2023.acl-long.74 2023

  40. [41]

    Wenhao Zhao, Shaoyang Song, and Chunlai Zhou. 2024. https://doi.org/10.24963/ijcai.2024/735 Generate synthetic text approximating the private distribution with differential privacy . In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24 , pages 6651--6659. International Joint Conferences on Artificial Intel...

    work page doi:10.24963/ijcai.2024/735 2024

  41. [42]

    Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. https://doi.org/10.1145/3209978.3210080 Texygen: A benchmarking platform for text generation models . In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR '18, page 1097–1100, New York, NY, USA. Association for Compu...

    arXiv 2018