pith. sign in

arxiv: 2604.23795 · v1 · submitted 2026-04-26 · 💻 cs.CR

LLM-CEG: Extending the Classification Error Gauge Framework for Privacy Auditing of Large Language Models

Kato Mivule This is my paper

Pith reviewed 2026-05-08 05:57 UTC · model grok-4.3

classification 💻 cs.CR
keywords privacy auditinglarge language modelsdifferential privacymembership inference attacksDP-SGDprivacy-utility tradeoffLLM fine-tuningclassification error gauge
0
0 comments X

The pith

The LLM-CEG framework shows differential privacy via DP-SGD can cut membership inference attacks by 71.5% while raising out-of-distribution utility by 47-50% in fine-tuned models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper extends the Classification Error Gauge framework from tabular data to large language models by creating LLM-CEG, which uses membership inference attack success as a privacy measure and perplexity as a utility measure. It iteratively tunes differential privacy parameters until both criteria are met. A prototype fine-tunes DistilGPT-2 on synthetic clinical data under DP-SGD, producing the reported gains over an overfitted baseline. The work also adapts the SIED process into LLM-SIED for auditable deployment. A sympathetic reader would care because it supplies a concrete method to audit and balance privacy against performance in models that process sensitive text.

Core claim

The central claim is that LLM-CEG provides a systematic way to audit LLMs by treating membership inference attack success rates as an empirical privacy gauge and model perplexity as a utility gauge, then adjusting differential privacy parameters until both thresholds are satisfied jointly; empirical results on DistilGPT-2 with a synthetic PII dataset demonstrate that DP-SGD achieves a 71.5% reduction in attacker advantage while improving out-of-distribution utility by 47-50% relative to the overfitted baseline, and that differential privacy can function as implicit regularization under narrow fine-tuning conditions.

What carries the argument

LLM-CEG, the iterative adjustment loop that uses membership inference attack success rates as the privacy gauge and model perplexity as the utility gauge to select differential privacy parameters for LLM fine-tuning.

If this is right

  • LLM fine-tuning can be audited for privacy compliance using existing MIA tools and perplexity calculations without new metrics.
  • Differential privacy may simultaneously improve generalization in narrow fine-tuning regimes rather than only trading off utility.
  • The LLM-SIED extension supplies a documented, regulator-aligned workflow for deploying privacy-compliant language models.
  • Model developers can search for joint privacy-utility operating points by varying DP-SGD noise and clipping parameters.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same auditing loop could be applied to larger foundation models or multimodal LLMs to check whether the regularization effect scales.
  • Regulators could adopt the LLM-SIED process as a template for mandatory privacy audits of deployed language models.
  • If the regularization benefit holds, practitioners might prefer DP-SGD over standard fine-tuning even when strict privacy budgets are not required.
  • Testing the framework with alternative privacy attacks beyond membership inference would strengthen its generality as an auditing tool.

Load-bearing premise

The assumption that membership inference attack success rates and model perplexity serve as reliable and sufficient proxies for privacy and utility in LLM settings, and that the observed benefits will hold outside the specific synthetic dataset and DistilGPT-2 model tested.

What would settle it

Repeating the fine-tuning experiments on a real-world PII dataset with a different LLM architecture and verifying whether the 71.5% reduction in MIA attacker advantage and 47-50% utility gain still appear.

Figures

Figures reproduced from arXiv: 2604.23795 by Kato Mivule.

Figure 1
Figure 1. Figure 1: shows where in the training pipeline this noise is actually applied. The proprietary dataset it￾self is never modified; DP-SGD operates strictly on the gradients computed from each mini-batch, form￾ing an “Opacus privacy wall” between raw learning 1. Proprietary data (untouched) ❄ 2. Tokenize & embed (still clean) ❄ 3. Compute per-sample gradients ❄ PRIVACY WALL (DP-SGD) 4a. Clip each gradient to norm C 4b… view at source ↗
Figure 2
Figure 2. Figure 2: provides a simplified view of the end-to-end LLM-CEG workflow, from data generation through iterative privacy-utility optimization to LLM-SIED dissemination. 1. Generate synthetic PII data ❄ 2. Fine-tune baseline (no privacy) ❄ 3. Fine-tune with DP-SGD (ε ∈ {8, 2, 0.5}) ❄ 4. Run membership inference attack (MIA) ❄ 5. Measure utility (perplexity) ❄ ✛ 6. Privacy AND utility thresholds met? ✲No: adjust ε ❄ Ye… view at source ↗
Figure 3
Figure 3. Figure 3: LLM-CEG Privacy-Utility Pareto Curve. Red line (left axis): MIA attacker advantage (lower is more private). Blue line (right axis): normalized utility score (higher is better). Green region: LLM-CEG acceptable operating zone (advantage ≤ 0.10). All three DP models satisfy the privacy threshold while exceeding the 100% utility baseline. DP ε=8 is the Pareto-optimal configura￾tion. ture of differential priva… view at source ↗
read the original abstract

This paper extends the Classification Error Gauge (x-CEG) framework, originally developed for measuring the privacy-utility trade-off in tabular datasets, to privacy auditing of Large Language Models (LLMs). We propose LLM-CEG, a systematic framework that employs membership inference attack (MIA) success rates as an empirical privacy gauge and model perplexity as a utility gauge, iteratively adjusting differential privacy parameters until both thresholds are jointly satisfied. A proof-of-concept prototype fine-tunes DistilGPT-2 on a synthetic clinical PII dataset under four privacy regimes using DP-SGD. Results indicate that DP-SGD reduces MIA attacker advantage by 71.5% while simultaneously improving out-of-distribution utility by 47-50% relative to the overfitted baseline, suggesting that differential privacy may act as implicit regularization under narrow fine-tuning conditions. We further extend the SIED engineering framework to the LLM context as LLM-SIED, providing an auditable, regulator-aligned process for privacy-compliant LLM deployment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript extends the x-CEG framework to LLMs as LLM-CEG, using membership inference attack (MIA) success rates as an empirical privacy gauge and model perplexity as a utility gauge. It iteratively tunes differential privacy parameters (via DP-SGD) until both thresholds are jointly satisfied. A proof-of-concept fine-tunes DistilGPT-2 on a synthetic clinical PII dataset, reporting that DP-SGD reduces MIA attacker advantage by 71.5% while improving out-of-distribution utility by 47-50% relative to an overfitted baseline; this is interpreted as evidence that differential privacy can act as implicit regularization under narrow fine-tuning. The work also extends the SIED framework to LLM-SIED for regulator-aligned, auditable LLM deployment.

Significance. If the reported privacy-utility gains hold under more rigorous validation, the LLM-CEG framework would provide a concrete, threshold-based auditing procedure for privacy-compliant LLM fine-tuning, which is currently lacking in the literature. The empirical observation that DP-SGD can simultaneously reduce MIA advantage and improve OOD perplexity challenges the conventional privacy-utility trade-off and could inform practical fine-tuning guidelines in sensitive domains. The LLM-SIED extension adds an engineering process dimension that aligns with regulatory needs.

major comments (2)
  1. [Abstract] Abstract and experimental evaluation: the headline quantitative claims (71.5% MIA attacker advantage reduction and 47-50% OOD utility improvement) are presented without any description of the MIA implementation details, threshold values chosen for LLM-CEG, statistical analysis, number of runs, or explicit baselines beyond the overfitted model, making it impossible to assess whether the data support the privacy-utility benefit or the implicit-regularization interpretation.
  2. [Experimental evaluation] Experimental evaluation: the central interpretation that differential privacy acts as implicit regularization rests on MIA success rate and perplexity being adequate proxies for LLM privacy (against extraction and memorization attacks) and utility; the manuscript provides no justification or sensitivity analysis for these proxies, nor any discussion of how the synthetic clinical PII dataset and DistilGPT-2 results would generalize to larger models or real data.
minor comments (2)
  1. [LLM-CEG Framework] The manuscript would benefit from a clear pseudocode or flowchart illustrating the iterative threshold-satisfaction procedure in LLM-CEG.
  2. [Introduction] Notation for the privacy and utility gauges should be defined consistently when first introduced.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback, which helps us improve the clarity and rigor of the manuscript. We address each major comment below and commit to revisions that strengthen the presentation of our proof-of-concept results.

read point-by-point responses

  1. Referee: [Abstract] Abstract and experimental evaluation: the headline quantitative claims (71.5% MIA attacker advantage reduction and 47-50% OOD utility improvement) are presented without any description of the MIA implementation details, threshold values chosen for LLM-CEG, statistical analysis, number of runs, or explicit baselines beyond the overfitted model, making it impossible to assess whether the data support the privacy-utility benefit or the implicit-regularization interpretation.

    Authors: We agree that the abstract requires additional context to make the quantitative claims assessable. In the revised manuscript we will expand the abstract to include: (i) a concise description of the MIA (shadow-model likelihood-ratio attack), (ii) the LLM-CEG threshold values used (MIA attacker advantage < 0.05 and OOD perplexity within 10 % of the non-private baseline), (iii) the fact that all metrics are means over five independent runs with standard deviations reported in the experimental section, and (iv) explicit identification of the baseline as the non-DP fine-tuned model that overfits the training data. These details already appear in Section 4; we will ensure they are summarized at the abstract level so readers can evaluate the privacy-utility claims and the implicit-regularization interpretation without needing to consult the full text first. revision: yes

  2. Referee: [Experimental evaluation] Experimental evaluation: the central interpretation that differential privacy acts as implicit regularization rests on MIA success rate and perplexity being adequate proxies for LLM privacy (against extraction and memorization attacks) and utility; the manuscript provides no justification or sensitivity analysis for these proxies, nor any discussion of how the synthetic clinical PII dataset and DistilGPT-2 results would generalize to larger models or real data.

    Authors: We accept that the manuscript should provide explicit justification for the chosen proxies and address generalization. We will add a new subsection in the methodology that justifies MIA success rate (as a direct empirical measure of membership leakage, consistent with prior LLM privacy literature) and perplexity (as the standard utility metric for autoregressive language models). We will also include a sensitivity analysis that varies the number of shadow models and attack hyperparameters to demonstrate robustness of the reported 71.5 % reduction. In the discussion and limitations section we will explicitly state that the work is a proof-of-concept on DistilGPT-2 with synthetic clinical PII data, discuss why results may differ for larger models or real-world datasets, and outline planned follow-up experiments. These additions will support rather than overstate the implicit-regularization observation. revision: yes

Circularity Check

0 steps flagged

No significant circularity; claims rest on empirical observations

full rationale

The paper proposes LLM-CEG as an extension of the prior CEG framework and reports direct experimental outcomes from fine-tuning DistilGPT-2 under DP-SGD on synthetic data, including measured reductions in MIA attacker advantage and improvements in out-of-distribution perplexity. These results are presented as observed values from the prototype runs rather than any mathematical derivation, fitted parameter, or prediction that reduces to its own inputs by construction. The iterative threshold-satisfaction procedure is a methodological description, and the interpretation of implicit regularization is a post-experiment suggestion, not a load-bearing self-referential step. No equations, ansatzes, or uniqueness claims appear that would trigger the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 0 invented entities

Since only the abstract is available, the ledger is based on high-level descriptions. The framework likely inherits assumptions from the original x-CEG and differential privacy literature.

free parameters (1)
  • DP parameters (epsilon, delta)
    The paper mentions iteratively adjusting differential privacy parameters but does not specify exact values or how they are chosen.
axioms (2)
  • domain assumption MIA success rate accurately measures privacy leakage
    Used as the privacy gauge without detailed justification in abstract.
  • domain assumption Perplexity accurately measures utility
    Used as the utility gauge.

pith-pipeline@v0.9.0 · 5468 in / 1440 out tokens · 97378 ms · 2026-05-08T05:57:11.299489+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages

  1. [1]

    An investigation of data privacy and utility using machine learning as a gauge,

    K. Mivule, “An investigation of data privacy and utility using machine learning as a gauge,” Doctoral Dissertation, Bowie State University, ProQuest UMI 3619387, 2014

    work page 2014

  2. [2]

    Utilizing noise addition for data privacy, an overview,

    K. Mivule, “Utilizing noise addition for data privacy, an overview,”arXiv:1309.3958, 2013

    work page arXiv 2013

  3. [3]

    Deep learning with differential privacy,

    M. Abadi et al., “Deep learning with differential privacy,”Proc. ACM CCS, pp. 308–318, 2016

    work page 2016

  4. [4]

    Differential privacy,

    C. Dwork, “Differential privacy,” inProc. ICALP, Springer, pp. 1–12, 2006

    work page 2006

  5. [5]

    Membership inference attacks against machine learning models,

    R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,”Proc. IEEE S&P, pp. 3–18, 2017

    work page 2017

  6. [6]

    Privacy risk in machine learning: Analyz- ing the connection to overfitting,

    S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyz- ing the connection to overfitting,”Proc. IEEE CSF, pp. 268–282, 2018

    work page 2018

  7. [7]

    Private empirical risk minimization,

    R. Bassily, A. Smith, and A. Thakurta, “Private empirical risk minimization,”J. ACM, vol. 63, no. 6, pp. 1–40, 2016

    work page 2016

  8. [8]

    Private stochastic convex optimization: Optimal rates inℓ 1 geom- etry,

    V. Feldman and T. Steinke, “Private stochastic convex optimization: Optimal rates inℓ 1 geom- etry,”Proc. ICML, pp. 3089–3098, 2020

    work page 2020

  9. [9]

    EW-Tune: A framework for privately fine-tuning large language models with differential privacy,

    R. Behnia, M. R. Ebrahimi, J. Pacheco, and B. Padmanabhan, “EW-Tune: A framework for privately fine-tuning large language models with differential privacy,”Proc. IEEE ICDMW, 2022

    work page 2022

  10. [10]

    PrivLLM-Guard: An adaptive differential privacy framework for clinical large language models,

    W. Alghamdi, “PrivLLM-Guard: An adaptive differential privacy framework for clinical large language models,”Scientific Reports, 2026

    work page 2026

  11. [11]

    Enhancing large lan- guage model privacy with differentially private parameter-efficient fine-tuning,

    T. Higashi and T. Nakai, “Enhancing large lan- guage model privacy with differentially private parameter-efficient fine-tuning,”LM-SHIELD Workshop, 2025

    work page 2025

  12. [12]

    Differential privacy in the era of large- scale generative AI,

    F. Wu, “Differential privacy in the era of large- scale generative AI,” Ph.D. Dissertation, Univ. of Illinois Urbana-Champaign, 2025

    work page 2025

  13. [13]

    Leveraging open LLMs for pri- vate adaptation without exposing data,

    F. Hanke et al., “Leveraging open LLMs for pri- vate adaptation without exposing data,”Proc. NeurIPS, 2024. 10

    work page 2024

  14. [14]

    Privacy-Flat: Towards flatter loss landscape for privacy-preserving large lan- guage models,

    Y. Chen et al., “Privacy-Flat: Towards flatter loss landscape for privacy-preserving large lan- guage models,”Proc. SIAM SDM, 2025

    work page 2025

  15. [15]

    Privacy in large language models: Attacks, defenses and future directions,

    H. Li et al., “Privacy in large language mod- els: Attacks, defenses, and future directions,” arXiv:2310.10383, 2023

    work page arXiv 2023

  16. [16]

    Privacy-Preserving4LLM: A benchmark for privacy-preserving techniques in large lan- guage model training,

    X. Sun, B. Suleiman, A. Ullah, and I. Raz- zak, “Privacy-Preserving4LLM: A benchmark for privacy-preserving techniques in large lan- guage model training,”Proc. ACM WWW, 2025

    work page 2025

  17. [17]

    Large language models are privacy-erasable,

    H. Ye and P. Luo, “Large language models are privacy-erasable,”Proc. EMNLP, 2025

    work page 2025

  18. [18]

    Towards a human-centered LLM privacy research agenda,

    T. Li et al., “Towards a human-centered LLM privacy research agenda,”CHI Extended Ab- stracts, 2024

    work page 2024

  19. [19]

    Human-centred privacy audits for large language models,

    L. Staufer, M. Morehouse, J. Hartmann, and B. Berendt, “Human-centred privacy audits for large language models,”CHI HEAL Workshop, 2026

    work page 2026

  20. [20]

    Human-centered privacy framework for AI systems,

    Y. Sun, J. Xu, and H. Gao, “Human-centered privacy framework for AI systems,”Springer Handbook of AI Ethics, 2026

    work page 2026

  21. [21]

    Human- centric AI: Philosophical foundations,

    M. Taylor, R. O’Dell, and S. Murphy, “Human- centric AI: Philosophical foundations,”AI & Society, 2024

    work page 2024

  22. [22]

    Security and privacy challenges of large language models,

    S. Das, M. H. Amini, and Y. Wu, “Security and privacy challenges of large language models,” ACM Computing Surveys, 2025

    work page 2025

  23. [23]

    DP-RAG: Applying differen- tial privacy to retrieval-augmented generation,

    N. Grislain, “DP-RAG: Applying differen- tial privacy to retrieval-augmented generation,” Proc. IEEE CAI, 2025

    work page 2025

  24. [24]

    Extracting training data from large language models,

    N. Carlini et al., “Extracting training data from large language models,”Proc. USENIX Secu- rity, pp. 2633–2650, 2021

    work page 2021

  25. [25]

    EU Artificial Intelli- gence Act,

    European Parliament, “EU Artificial Intelli- gence Act,” Regulation (EU) 2024/1689, 2024

    work page 2024

  26. [26]

    AI Risk Management Framework,

    NIST, “AI Risk Management Framework,” NIST AI 100-1, 2023. 11

    work page 2023