Pith. sign in

REVIEW 2 major objections 4 minor 2 cited by

DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators

T0 review · 2 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Two-stage tuning — learn the table template from pseudo data first, then fine-tune under DP on the private data — improves the utility and fidelity of privately generated tabular data over direct DP fine-tuning.

desk verdict Solid empirical analysis of two-stage DP fine-tuning, but the headline claim only holds for the O variant and the U variant is not DP as presented. read the letter →

arxiv 2412.02467 v2 pith:UGFRLIPA submitted 2024-12-03 cs.LG cs.CLcs.CR

Tejumade Afonja , Hui-Po Wang , Raouf Kerkouche , Mario Fritz This is my paper · ORCID
classification cs.LGcs.CLcs.CR
keywords differentialprivacysynthetictabulardatalargelanguagemodelsDP-SGDtwo-stagefine-tuningpseudodatasetgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Directly fine-tuning a language model under differential privacy to generate tabular data spends much of the privacy budget on tokens that are not private — column names and filler words such as 'is' and ',' — which is why the output loses coherence. The paper proposes DP-2Stage, a two-stage fine-tuning schedule in which the model first learns the table's serialization template from a non-private pseudo dataset, and then is fine-tuned on the real private data with DP-SGD while a weighted loss emphasizes value tokens. On the Adult, Airline, and Texas datasets at $\varepsilon = 1$, the two-stage models improve F1 by 12–25% and marginal-distribution fidelity by 1–3% over directly fine-tuned DP models, and the uniform-pseudo variant generates a synthetic dataset up to 21× faster. The authors also state that only the out-of-distribution pseudo-data variant carries the full privacy guarantee as published, because the uniform variant builds its pseudo data from private column statistics.

What carries the argument

The load-bearing object is the pseudo dataset: a non-private table that instantiates the same '<key> is <value>,' serialization template as the private data, so the model can absorb the template in stage one without consuming the stage-two privacy budget (the paper flags that the uniform variant's statistics still touch private data). The second load-bearing mechanism is the stage-two weighted loss, Equation (6), which places weight $\lambda = 0.65$ on value-token log-likelihoods and weight $1-\lambda$ on key and filler tokens, steering the DP-SGD noise toward private content. Together these convert the privacy budget from a tax on the whole sentence into a tax on values only.

What would settle it

Rerun DP-2Stage-U's stage one with its column-range and category-list statistics replaced by differentially private versions of the same statistics and recompute the reported $\varepsilon$: if the effective privacy loss at $\delta = 10^{-5}$ exceeds the claimed $\varepsilon = 1$, the uniform variant as presented fails its own privacy claim. Separately, to test the structural-transfer story, feed DP-2Stage-O a stage-one corpus of randomly ordered non-tabular text with the same serialization template; if the F1 and Hist gains over DP-Standard vanish, the benefit is the template-learning mechanism and not merely extra fine-tuning steps.

Watch

Extended reading notes

Core claim

The paper's central claim is that LLM-based tabular synthesis under DP fails not because the model cannot learn private values but because DP-SGD's noise is spilled on learning the table's structure — keys, 'is' tokens, commas — which are not private. DP-2Stage therefore decouples the two tasks: stage one fine-tunes a pre-trained GPT-2 on a pseudo dataset, built either by sampling uniformly from private column ranges and category lists (DP-2Stage-U) or by taking an unrelated public dataset (DP-2Stage-O), so that template learning consumes none of the stage-two privacy budget; stage two fine-tunes the result with DP-SGD on the private data using a loss weighted with $\lambda = 0.65$ toward value tokens. Across three datasets at $\varepsilon = 1$, $\delta = 10^{-5}$, both variants match or exceed direct DP fine-tuning on utility and fidelity, with DP-2Stage-O best on Adult and Texas, DP-2Stage-U best among LLMs on Airline and fastest at sampling, and the paper explicitly conceding that DP-2Stage-U as presented is not fully private because its stage-one statistics come from the private data.

Load-bearing premise

For the out-of-distribution variant, the load-bearing premise is that structure learned from an unrelated public table transfers to the private table; for the uniform variant, the load-bearing premise is that the private column ranges and category lists used to build the pseudo data leak nothing — a premise the authors themselves say is false, so only the out-of-distribution variant carries the stated $(\varepsilon,\delta)$ guarantee as published.

Editorial extensions

If this is right

  • DP fine-tuning of LLMs for tabular data should spend its budget on values, not on template tokens, and the two-stage schedule is a way to do that.
  • Using an out-of-distribution public dataset as pseudo data keeps the method strictly DP as stated and still competitive, outperforming direct DP fine-tuning on Adult and Texas.
  • Using uniform pseudo data built from private column statistics makes generation up to 21× faster, which is the practical advantage of the variant if its privacy gap is closed.
  • Disabling column shuffling helps every DP variant at ε = 1, the opposite of the non-DP convention.
  • Raising the budget to ε = 8 improves DP-2Stage utility, consistent with the noise-allocation diagnosis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Read literally, DP-2Stage-U's results are what a fully private pipeline could hope to achieve only after its stage-one statistics are themselves made private; a version that computes column ranges and category lists under DP is the natural next test, and the paper does not run it.
  • The structure-then-content split should transfer beyond tables: DP fine-tuning of generative models over any rigidly structured output — JSON, code, schemas, forms — plausibly wastes noise on syntax, so a pseudo-data warm-up is a cheap experiment in those settings.
  • The finding that column shuffling hurts DP but helps non-DP training suggests order noise and gradient noise interact super-linearly, and understanding that interaction could yield DP-aware data-loading rules rather than simply disabling shuffling.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes DP-2Stage, a two-stage fine-tuning framework for LLM-based differentially private tabular data generation. The first stage non-privately fine-tunes a pre-trained LLM on a pseudo dataset, constructed either by uniform sampling from private column statistics (DP-2Stage-U) or from an out-of-distribution public dataset (DP-2Stage-O). The second stage applies DP-SGD on the private dataset. The authors report improved F1, histogram intersection, and perplexity compared with direct DP fine-tuning (DP-Standard) on Adult, Airline, and Texas datasets, and they release code and experimental setup.

Significance. If the central claim held, the two-stage approach would offer a practical way to redirect the DP budget from table structure toward private values, which is a plausible and useful idea for LLM-based tabular synthesis. The paper is empirically grounded: it reports five runs with standard deviations, compares against several baselines, and includes a limitations section that openly discusses open problems. However, the privacy guarantee for DP-2Stage-U is not end-to-end as presented, and the utility improvements over DP-Standard are inconsistent across datasets and variants. These issues weaken the paper's headline contribution as currently stated, although the underlying framework remains of interest if the claims are appropriately scoped.

major comments (2)
  1. [Section 4.1 and Section 6] DP-2Stage-U as presented is not (epsilon, delta)-differentially private. In Section 4.1, the uniform pseudo dataset is constructed using the range of numerical columns and the category lists of the private dataset, and this construction is not protected by any DP mechanism. Section 6 explicitly states that this 'weakens the stated privacy guarantees' and recommends that the statistics be computed in a differentially private manner. Therefore, the end-to-end (epsilon, delta)-DP guarantee of Definition 3.2 does not hold for DP-2Stage-U as evaluated in Table 3. The abstract and the contributions list DP-2Stage-U as part of the proposed framework without this caveat, which overstates the privacy property. The paper should either compute the pseudo-data statistics with a dedicated privacy budget, or clearly present DP-2Stage-U as a non-DP variant and restrict the privacy claim to DP-2Stage-O.
  2. [Abstract and Table 3] The claim that the approach 'improves performance across various settings and metrics' is not consistently supported by the reported results. In Table 3, on Adult, DP-2Stage-U is worse than DP-Standard on F1 (21.2 vs 27.8), AUC (48.9 vs 58.5), and ACC (61.9 vs 65.2). On Airline, DP-2Stage-O (+adult) is worse than DP-Standard on F1 (55.2 vs 60.5), AUC (62.5 vs 65.3), and ACC (60.0 vs 62.4). On Texas, DP-2Stage-U is substantially worse than DP-Standard on F1 (23.5 vs 55.4). The gains are clear only for DP-2Stage-O on Texas and partially on Adult, and for DP-2Stage-U on Airline. The paper should either revise the central claim to specify the settings in which improvements occur, or provide an analysis of the conditions under which the two-stage approach helps or hurts.
minor comments (4)
  1. [Abstract] There is a grammatical error: 'Our findings shows' should be 'Our findings show.'
  2. [Section 5.1] The text says 'the size of the synthethic data is the same' and later the same spelling appears; 'synthethic' should be 'synthetic.'
  3. [Table 4] In Table 4, DP-Standard and DP-2Stage-O with column shuffling are reported from a single model run, while other configurations use five runs. The absence of standard deviations for those entries is acknowledged in Section 5.4.1, but the visual formatting makes it easy to misread all entries as comparable. Adding a note or marker directly in the table would improve clarity.
  4. [Section 5.4.2] The discussion of lambda weighting states that DP-2Stage-U generally performs better with the default loss on Adult, but the table shows mixed results; a more precise summary of the pattern would help the reader.

Circularity Check

0 steps flagged · score 1.0 of 10

No material circularity: the two-stage utility gain is established by an internal comparison against DP-Standard under identical protocols, and the flagged Section 6 limitation is a privacy soundness gap, not a by-construction equivalence.

full rationale

The central claim, that non-private stage-1 fine-tuning on pseudo data followed by DP fine-tuning improves utility and fidelity over direct DP fine-tuning, is an empirical result tested inside the paper against an internal baseline (DP-Standard) under identical GPT-2/Opacus training protocols (10 epochs, same LR, same DP-SGD accounting). No load-bearing step reduces to its own inputs by construction: the stage-2 loss weight lambda=0.65 is adopted from external concurrent work Tran & Xiong (2024) and is ablated against the default lambda=0.33 in Table 5 rather than fitted to the reported metrics; stage-1 duration of 5 epochs is likewise taken from that concurrent work; and the stage-1 pseudo datasets are constructed from public data (DP-2Stage-O) or from private column ranges and category lists (DP-2Stage-U), while all reported F1/AUC/ACC/CorAcc/Pair/Hist values are measured on model-generated samples rather than read off the pseudo data. The self-citations that exist (Afonja et al., 2023 for the Texas binary-classification framing and the CorAcc/Pair metrics; Wang & Fritz, 2024 as a general LLM-capability reference) are not load-bearing, because the contribution is supported by the paper's own Table 3 comparisons, not by those citations. The reviewing rule requires explicit weighing of the paper's own flagged limitation: Section 6 states that 'the proposed DP-2Stage-U method utilizes private data to compute statistics for the uniformly sampled dataset, which weakens the stated privacy guarantees' and that these statistics 'should instead be computed in a differentially private manner.' This is a genuine soundness gap, meaning the (epsilon,delta)-DP guarantee of Definition 3.2 does not hold end-to-end for DP-2Stage-U as presented, and the abstract's unqualified privacy framing plus the DP-2Stage-U rows of Table 3 must be read with that caveat. Relatedly, the abstract's claim that the approach 'improves performance across various settings and metrics' is overstated, because Table 3 shows DP-2Stage-U below DP-Standard on all utility metrics for Adult and Texas and DP-2Stage-O below DP-Standard on Airline. Both issues are correctness/robustness concerns, not instances of fitted inputs being renamed as predictions or of definitions entailing the target result, so they do not raise the circularity score.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on standard DP-SGD privacy accounting and on the transferability of stage-1 structure learning. DP-2Stage-U additionally depends on the assumption that private column ranges and category lists can be used without breaking DP, which the paper itself concedes. No new physical or mathematical entities are introduced.

free parameters (3)
  • lambda (stage-2 value-token loss weight) = 0.65
    Set consistently in Section 5.1; Table 5 shows the optimal value is dataset-dependent and gains for DP-2Stage-U are inconsistent.
  • Private column ranges and category lists = per dataset, not fully reported
    Used to build uniform pseudo data in Section 4.1; these are data-derived inputs and the reason DP-2Stage-U is not strictly DP.
  • Stage-1 epochs = 5
    Adopted in Section 5.1 and listed in Limitations as unexplored; affects how much structure is learned before DP training.
assumptions (4)
  • standard math DP-SGD with RDP accounting provides valid (epsilon, delta)-DP for the second-stage fine-tuning.
    Invoked in Section 5.1 and Appendix A.2 via Abadi et al. 2016, Mironov et al. 2019, and Balle et al. 2020.
  • domain assumption GReaT serialization ('key is value,') preserves the information needed for tabular generation.
    Used throughout; column-shuffling analysis in Section 5.4.1 assumes this template is the right input format.
  • domain assumption Structure learned on pseudo data in stage 1 transfers to the private table in stage 2.
    Section 4.2 and Figure 3 rest on this transfer; results are dataset-dependent, and Section 5.4.3 hypothesizes overfitting for DP-2Stage-U.
  • ad hoc to paper Column ranges and category lists are not private or are safe to use without DP.
    Needed for DP-2Stage-U in Section 4.1; Section 6 acknowledges this weakens the privacy guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators." pith.science (2026). https://pith.science/paper/UGFRLIPA

@misc{pith2026241202467,
  author       = {Pith},
  title        = {Pith review of: DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UGFRLIPA}},
  note         = {Machine review of arXiv:2412.02467}
}
read the original abstract

Generating tabular data under differential privacy (DP) protection ensures theoretical privacy guarantees but poses challenges for training machine learning models, primarily due to the need to capture complex structures under noisy supervision signals. Recently, pre-trained Large Language Models (LLMs) -- even those at the scale of GPT-2 -- have demonstrated great potential in synthesizing tabular data. However, their applications under DP constraints remain largely unexplored. In this work, we address this gap by applying DP techniques to the generation of synthetic tabular data. Our findings shows that LLMs face difficulties in generating coherent text when fine-tuned with DP, as privacy budgets are inefficiently allocated to non-private elements like table structures. To overcome this, we propose DP-2Stage, a two-stage fine-tuning framework for differentially private tabular data generation. The first stage involves non-private fine-tuning on a pseudo dataset, followed by DP fine-tuning on a private dataset. Our empirical results show that this approach improves performance across various settings and metrics compared to directly fine-tuned LLMs in DP contexts. We release our code and setup at https://github.com/tejuafonja/DP-2Stage.

Figures

Figures reproduced from arXiv: 2412.02467 by the authors.

Figure 1
Figure 1. Overview of DP-2Stage. In stage 1, the pre-trained LLM is fine-tuned on the respective pseudo data. Subsequently, in stage 2, the model from stage 1 undergoes further fine-tuning using the real private data. LLM-based tabular data generators. To support future advancements, we release our code and provide a detailed discussion to guide further investigation. 2 Related Work Tabular data generation. As a prominent sol… view at source ↗
Figure 2
Figure 2. Illustration of column shuffling. The order of entries is permuted in each iteration. This mechanism happens at every iteration and has been shown to effectively prevent the model from relying on spurious dependency in Non-DP settings (Borisov et al., 2023). However, we find that it complicates DP training due to gradient perturbations, often resulting in higher perplexity compared to Non-DP models, as shown in [PI… view at source ↗
Figure 3
Figure 3. DP-2Stage (Ours) vs. Standard DP fine-tuning on the Adult dataset with ε = 1, δ = 10−5 . DP-2Stage-O refers to the stage 2 model fine-tuned using out-distribution pseudo data (Airline dataset) in stage 1, while DP-2Stage-U is the stage 2 model fine-tuned using data sampled independently from a Uniform distribution, with statistics derived from the Adult dataset as the pseudo-dataset in stage 1. Perplexity results ar… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Struct-Bench: A Benchmark for Differentially Private Structured Text Generation

    cs.CL 2025-09 conditional novelty 7.0 of 10

    Struct-Bench is a Context-Free Grammar based evaluation platform showing that current differentially private synthetic text generators often fail to preserve structure and semantic diversity.

  2. Is API Access to LLMs Useful for Generating Private Synthetic Tabular Data?

    cs.LG 2025-02 conditional novelty 6.0 of 10

    API access to Gemini 1.0 Pro does not improve differentially private synthetic tabular data beyond established non-LLM baselines on the tested datasets and workloads.

Reference graph

Works this paper leans on

43 extracted references · 33 canonical work pages · cited by 2 Pith papers

  1. [1]

    Deep learning with differential privacy

    Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2016

    work page 2016

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    marginally

    Tejumade Afonja, Dingfan Chen, and Mario Fritz. Margctgan: A “marginally” better ctgan for the low sample regime. In German Conference on Pattern Recognition (GCPR), 2023

    work page 2023

  4. [4]

    Generating synthetic but plausible healthcare record datasets

    Laura Avi \ n \'o , Matteo Ruffini, and Ricard Gavald \`a . Generating synthetic but plausible healthcare record datasets. KDD workshop on Machine Learning for Medicine and Healthcare, 2018

    work page 2018

  5. [5]

    Hypothesis testing interpretations and renyi differential privacy

    Borja Balle, Gilles Barthe, Marco Gaboardi, Justin Hsu, and Tetsuya Sato. Hypothesis testing interpretations and renyi differential privacy. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), 2020

    work page 2020

  6. [6]

    Language models are realistic tabular data generators

    Vadim Borisov, Kathrin Sessler, Tobias Leemann, Martin Pawelczyk, and Gjergji Kasneci. Language models are realistic tabular data generators. In Proceedings of the International Conference on Learning Representations (ICLR), 2023

    work page 2023

  7. [7]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS), 2020

    work page 2020

  8. [8]

    Gan-leaks: A taxonomy of membership inference attacks against generative models

    Dingfan Chen, Ning Yu, Yang Zhang, and Mario Fritz. Gan-leaks: A taxonomy of membership inference attacks against generative models. In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2020

    work page 2020

Show all 43 references
  1. [9]

    Generating multi-label discrete patient records using generative adversarial networks

    Edward Choi, Siddharth Biswal, Bradley Malin, Jon Duke, Walter F Stewart, and Jimeng Sun. Generating multi-label discrete patient records using generative adversarial networks. In Machine learning for healthcare conference, 2017

    2017

  2. [10]

    Scaling instruction-finetuned language models

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research (JMLR), 2022

    2022

  3. [11]

    Differential privacy

    Cynthia Dwork. Differential privacy. In International colloquium on automata, languages, and programming, 2006

    2006

  4. [12]

    The algorithmic foundations of differential privacy

    Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science , 2014

    2014

  5. [13]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Neural Information Processing Systems (NeurIPS), 2014

    2014

  6. [14]

    Monte carlo and reconstruction membership inference attacks against generative models

    Benjamin Hilprecht, Martin H \"a rterich, and Daniel Bernau. Monte carlo and reconstruction membership inference attacks against generative models. Proc. Priv. Enhancing Technol., 2019

    2019

  7. [15]

    Long short-term memory

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 1997

    1997

  8. [16]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2021

    2021

  9. [17]

    Statistical methods for speech recognition

    Frederick Jelinek. Statistical methods for speech recognition. MIT press, 1998

    1998

  10. [18]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. Proceedings of the International Conference on Learning Representations (ICLR), 2014

    2014

  11. [19]

    Tabddpm: Modelling tabular data with diffusion models

    Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. Tabddpm: Modelling tabular data with diffusion models. In Proceedings of the International Conference on Machine Learning (ICML), 2023

    2023

  12. [20]

    Large language models can be strong differentially private learners

    Xuechen Li, Florian Tram \`e r, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. In International Conference on Learning Representations (ICLR), 2022

    2022

  13. [21]

    Winning the nist contest: A scalable and general approach to differentially private synthetic data

    Ryan McKenna, Gerome Miklau, and Daniel Sheldon. Winning the nist contest: A scalable and general approach to differentially private synthetic data. Journal of Privacy and Confidentiality, 2021

    2021

  14. [22]

    Aim: an adaptive and iterative mechanism for differentially private synthetic data

    Ryan McKenna, Brett Mullins, Daniel Sheldon, and Gerome Miklau. Aim: an adaptive and iterative mechanism for differentially private synthetic data. Proceedings of the VLDB Endowment, 2022

    2022

  15. [23]

    Rényi differential privacy

    Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th computer security foundations symposium (CSF). IEEE, 2017

    2017

  16. [24]

    Rényi differential privacy of the sampled gaussian mechanism

    Ilya Mironov, Kunal Talwar, and Li Zhang. Rényi differential privacy of the sampled gaussian mechanism. arXiv preprint arXiv:1908.10530, 2019

    1908 arXiv

  17. [25]

    Data synthesis based on generative adversarial networks

    Noseong Park, Mahmoud Mohammadi, Kshitij Gorde, Sushil Jajodia, Hongkyu Park, and Youngmin Kim. Data synthesis based on generative adversarial networks. Proc. VLDB Endow., 2018

    2018

  18. [26]

    How to dp-fy ml: A practical guide to machine learning with differential privacy

    Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin, Zheng Xu, Carson Denison, H Brendan McMahan, Sergei Vassilvitskii, Steve Chien, and Abhradeep Guha Thakurta. How to dp-fy ml: A practical guide to machine learning with differential privacy. Journal of Artificial Intelligence ...

    2023

  19. [27]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 2019

    2019

  20. [28]

    Neural machine translation of rare words with subword units

    Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909, 2015

    2015 arXiv

  21. [29]

    Membership inference attacks against machine learning models

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP). IEEE, 2017

    2017

  22. [30]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the International Conference on Machine Learning (ICML), 2015

    2015

  23. [31]

    Synthetic data--anonymisation groundhog day

    Theresa Stadler, Bristena Oprisanu, and Carmela Troncoso. Synthetic data--anonymisation groundhog day. In 31st USENIX Security Symposium (USENIX Security 22), 2022

    2022

  24. [32]

    Benchmarking differentially private synthetic data generation algorithms

    Yuchao Tao, Ryan McKenna, Michael Hay, Ashwin Machanavajjhala, and Gerome Miklau. Benchmarking differentially private synthetic data generation algorithms. arXiv preprint arXiv:2112.09238, 2021

    2021 arXiv

  25. [33]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  26. [34]

    Differentially private tabular data synthesis using large language models

    Toan V Tran and Li Xiong. Differentially private tabular data synthesis using large language models. arXiv preprint arXiv:2406.01457, 2024

    2024 arXiv

  27. [35]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  28. [36]

    Language models as zero-shot lossless gradient compressors: Towards general neural parameter prior models

    Hui-Po Wang and Mario Fritz. Language models as zero-shot lossless gradient compressors: Towards general neural parameter prior models. Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  29. [37]

    Finetuned language models are zero-shot learners

    Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. In Proceedings of the International Conference on Learning Representations (ICLR), 2021

    2021

  30. [38]

    Differentially private generative adversarial network

    Liyang Xie, Kaixiang Lin, Shu Wang, Fei Wang, and Jiayu Zhou. Differentially private generative adversarial network. arXiv preprint arXiv:1802.06739, 2018

    2018 arXiv

  31. [39]

    Modeling tabular data using conditional gan

    Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. Modeling tabular data using conditional gan. Advances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  32. [40]

    Differentially private fine-tuning of language models

    Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. Differentially private fine-tuning of language models. In Proceedings of the International Conference on Learning Represen...

    2021

  33. [41]

    Privbayes: Private data release via bayesian networks

    Jun Zhang, Graham Cormode, Cecilia M Procopiuc, Divesh Srivastava, and Xiaokui Xiao. Privbayes: Private data release via bayesian networks. ACM Transactions on Database Systems (TODS), 2017

    2017

  34. [42]

    Large language models are human-level prompt engineers

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. In Proceedings of the International Conference on Learning Representations (ICLR), 2022

    2022

  35. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.