Pith. sign in

REVIEW 5 major objections 5 minor 58 references

LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LoRA fine-tuning does not make training data safe: using the public pre-trained model as a reference, membership inference reaches 0.775 AUC.

desk verdict A careful empirical study that shows LoRA fine-tuning is not MIA-immune; the pt-reference calibration is the real contribution, but the paper needs error bars, code, and a cleaner mechanism story. read the letter →

arxiv 2507.18302 v1 pith:YFBZ4OHU submitted 2025-07-24 cs.CR cs.AIcs.CL

Delong Ran , Xinlei He , Tianshuo Cong , Anyu Wang , Qi Li , Xiaoyun Wang This is my paper · ORCID
classification cs.CRcs.AIcs.CL
keywords membershipinferenceLoRAfine-tuninglanguagemodelprivacypre-trainedreferenceparameter-efficientleakagelow-rankadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LoRA fine-tuning updates only a small set of low-rank matrices, which has led to the impression that the fine-tuning data is safe from membership inference. This paper argues the opposite: an attacker who holds both the fine-tuned LoRA model and the public pre-trained model it was built from can reliably detect fine-tuning members, with the best attack reaching 0.775 AUC under a deliberately mild three-epoch setting. The key move is to use the pre-trained model as a reference and subtract its score from the fine-tuned model's score, highlighting the change caused by fine-tuning. If true, the result matters because LoRA is the most common way specialized language models are released, and the pre-trained base model is always publicly known. The paper also identifies practical mitigations: dropout and skipping certain LoRA layers reduce the risk, while weight decay does not.

What carries the argument

The central object is pre-trained-model calibration, defined as $S_{\mathrm{pt-ref}}(x;M) = S(x;M_{\mathrm{pt}}) - S(x;M)$, where $M_{\mathrm{pt}}$ is the public pre-trained model and $S$ is any membership score such as loss, Min-K% token probabilities, neighborhood losses, perturbed-model losses, or embedding gradients. The subtraction uses the base model as a natural shadow model: it estimates how surprising the sample was before fine-tuning, so the residual captures the memorization introduced by LoRA. This identity carries the argument because it upgrades five existing attacks with no extra training data, no shadow models, and no additional cost beyond having the base model, which is always accessible in LoRA deployments.

What would settle it

Run the best calibrated Min-K%++ attack against a LoRA model fine-tuned on a random sample drawn from the base model's own pretraining corpus. If the AUC drops from about 0.77 toward 0.5 while non-referenced attacks stay high, the apparent leakage is base-model memorization, not LoRA memorization.

Watch

Extended reading notes

Core claim

The paper establishes that the small number of trainable parameters in LoRA does not protect fine-tuning data from membership inference. Using the publicly available pre-trained base model as a reference, the attack score becomes a difference between the score on the base model and the score on the fine-tuned model, which isolates what fine-tuning actually changed. In the paper's three-epoch, low-overfitting setting, the best calibrated attack reaches 0.775 AUC on a medical QA task, and calibration consistently beats uncalibrated baselines. The paper also reports that LoRA fine-tuning is more vulnerable when certain feed-forward layers are trained, that weight decay does not stem the leakage, and that dropout or excluding the up and gate layers reduces it while retaining most utility.

Load-bearing premise

The load-bearing premise is that the public pre-trained base model has not already memorized the fine-tuning samples; if it has, subtracting its score from the tuned model's score cancels the membership signal and the five calibrated attacks collapse.

Editorial extensions

If this is right

  • Releasing a LoRA adapter on a public base model exposes the fine-tuning set to identification even when the model is only lightly fine-tuned.
  • An attacker needs no extra data or shadow models: with white-box access to the adapter and the public base model, every calibrated attack can be run directly.
  • Token-level confidence, as scored by Min-K%++ with base-model calibration, is the strongest single membership signal in the evaluation.
  • Which LoRA modules get trained changes the risk: excluding the up-projection (and, for Llama-2, the gate) layers lowers the best AUC by about 0.12 to 0.15 while keeping perplexity close to the full-module setting.
  • Practical defenses are dropout at high rates and layer exclusion; weight decay does not reduce leakage, and differential privacy eliminates it only at severe utility and runtime cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper: a model provider with a labeled holdout could use the calibrated scores as a pre-release privacy audit, adjusting dropout or module selection until the attack AUC crosses a chosen threshold.
  • Going beyond the paper: the module-level finding suggests the up-projection stores a disproportionate share of fine-tuning-specific knowledge; if so, adapters that skip that layer may leak through a different signal than loss or token confidence.
  • Going beyond the paper: the calibration arithmetic is not specific to the six scores tested, so any score with a meaningful value on both the base model and the tuned model should inherit the same boost, which likely transfers to other parameter-efficient fine-tuning methods built on public base models.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents LoRA-Leak, a framework that organizes 15 membership inference attacks (MIAs) against LoRA fine-tuned language models, of which five are new variants that use the publicly available pre-trained base model as a reference to calibrate the attack score via Eq. (13). The authors evaluate the framework on three base models (GPT-2 XL, Pythia-2.8B, Llama-2 7B) and three datasets (AG News, OAsst, MedQA) under a random membership split and a conservative 3-epoch fine-tuning setting, reporting best AUCs up to 0.775 (Min-K%++ with pre-trained reference on MedQA/Llama-2). They also analyze the impact of fine-tuning module selection, LoRA variants, model scale, and four defenses, finding that dropout and excluding the up/gate layers are effective mitigations while preserving utility.

Significance. The paper addresses a timely and practically important question with a careful empirical setup: it uses disjoint random member/non-member splits, multiple model families and datasets, and a deliberately low-overfit training setting, and it makes a useful observation that the public base model is a free and effective reference for MIA. If the results are confirmed, the work would refute the impression that LoRA fine-tuning is inherently resistant to membership inference and would provide actionable guidance (dropout, module exclusion) for practitioners. The main limitations are the ambiguity in the definitions of the new attacks, the lack of uncertainty quantification, the unexplained omission of one advertised attack (SPV), and the absence of a control isolating the base-model effect from text-level confounds.

major comments (5)
  1. [Section IV-C, Eq. (13); Section IV-B, Eqs. (4)-(9)] The paper does not specify how the pre-trained-model calibration is instantiated for the five proposed attacks. For Min-K% and Min-K%++ defined in Eqs. (4)-(5), the token subset Min-K%(x) is model-dependent, so S(x;Mpt) and S(x;M) may average over different token sets, and Eq. (13) does not define a token-aligned comparison. For Neighborhood (Eq. 8) and MoPe (Eq. 9), it is unspecified whether the paraphrased samples or the Gaussian parameter perturbations are shared between Mpt and Mft, nor how S(x;Mpt) is computed for these scores. Please provide precise algorithmic definitions for all five pt-referenced attacks; without them the reported improvements in Table II and Section V-B are not reproducible and could be sensitive to arbitrary implementation choices.
  2. [Section V-B, Table II, Figure 3 and Abstract] The headline AUCs are selected as the best among fourteen attacks on the same test set, and no runs across random seeds or confidence intervals are reported. Since the best-of-many statistic is an order statistic, it is biased upward relative to the performance of any individual attack; the 0.775 figure in the abstract is one such maximum. Please report mean and standard deviation (or bootstrap confidence intervals) over multiple fine-tuning/test-split seeds, and state in the abstract and takeaways that the reported value is the best over the attack suite.
  3. [Section IV-C and Section V-C] The claim that the Mpt-referenced calibration reveals LoRA-induced leakage is not supported by a control that isolates the effect of using the base model as the reference. Because members and non-members are both drawn from the same domain, a constant domain prior would cancel, but S(x;Mpt) may still correlate with text-level properties (length, token frequency, format) that differ between the random member/non-member splits, so the subtraction can change rankings without reflecting fine-tuning memorization. The comparison against shadow and self-prompt references in Figure 4 does not rule this out, since those references are also derived from the fine-tuning domain. Please add (i) a permutation test in which membership labels are shuffled and the pt-referenced AUC distribution is reported, and/or (ii) an evaluation with an unrelated reference model (e.g., a different pre-trained base that was not used for fine-tuning) to confirm that the gain is specific to the actual base model.
  4. [Table I and Section V] The paper advertises a framework of fifteen attacks, but SPV (described in Section IV-B, Eq. (12), and listed in Table I) is never evaluated in any of the reported experiments (Tables II and IX, Figures 3-10). Either evaluate SPV in the main comparison or explicitly state that it is omitted from the experiments and explain why; the count of evaluated attacks should be corrected accordingly.
  5. [Section V-A] Attack-specific hyperparameters are not reported: the K percentage for Min-K% and Min-K%++, the number of paraphrases N and the mask-filling model for the Neighborhood attack, the noise scale and number of perturbations for MoPe, and the exact norm used for GradNormx are all missing. This prevents reproduction and makes it hard to assess whether the relative performance of attacks is attributable to the attack design or to specific parameter choices.
minor comments (5)
  1. [Section II-C and Eq. (13)] The score functions are not consistently oriented so that higher values indicate membership; for example, SMin-K% (Eq. 4) is a negative log-likelihood and is lower for members, while SLOSS (Eq. 3) is higher for members. Applying Eq. (13) uniformly without noting that the threshold direction must be inverted per attack is confusing; please state the sign convention or note that AUC is invariant to monotone transformations.
  2. [Table II] The arrow notation in the table (e.g., 'LOSS 0.648 0.530 0.600' followed by '+Pre 0.705 0.583 0.609') is not explained; please define it in the caption, for example as 'non-referenced AUC followed by pt-referenced AUC'.
  3. [Section V-B] The text says 'we report the best AUC among the eight non-referenced MIAs and six referenced MIAs' for Figure 3, but Table II reports per-attack values; please clarify that the 'best AUC' series in Figure 3 is the maximum over the attack set at each epoch.
  4. [Section III-C] The statement that current inference APIs 'do not provide any internal states that aligns their assumption' is imprecise, since some APIs return token log-probabilities; please clarify which internal states (loss, gradients, embedding-level quantities) are unavailable, as opposed to output log-probabilities.
  5. [Abstract and Table I] The abstract counts 'ten existing MIAs and five improved MIAs' while the evaluation appears to cover eight non-referenced and six referenced attacks (SPV excluded); please harmonize the counting so that the abstract, Table I, and the experimental section agree.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the pt-reference attacks are parameter-free score combinations evaluated against independently labeled members and non-members, and the claimed amplification is an empirical result rather than an identity.

full rationale

I examined the derivation chain in Sections IV-B, IV-C, and V-B. The core proposed mechanism, Spt-ref(x;M) = S(x;Mpt) - S(x;M) (Eq. 13), is a new score function constructed from the fine-tuned model and its pre-trained reference. It contains no fitted constants, no threshold tuned on the membership labels, and no parameter estimated from the attack targets; the AUC values in Table II and Table IX are computed from these scores against independently sampled members and non-members. The claim that S(x;Mpt) estimates an a priori membership probability is an informal interpretation, not a premise that forces the reported amplification: if the baseline were uninformative, the measured AUC would simply fail to improve. The five 'improved' MIAs are exactly the named existing score functions (Min-K%, Min-K%++, GradNormx, Neighborhood, MoPe) combined with Eq. (13); the paper does not redefine those scores in terms of the membership outcome, so the improvement is an empirical comparison, not a tautology. No load-bearing self-citation is present: the closest prior attack, the reference attack [36], is cited as prior art and is not authored by the present authors, and the paper's contribution is extending that idea to additional internal states. The practice of reporting the best AUC among fifteen attacks without error bars is a methodological robustness concern, not circularity. I find no equation that is equivalent to its inputs by construction and no fitted input renamed as a prediction.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The ledger is light: no fitted free parameters. The method relies on standard MIA score definitions and on the untested separability assumption of Eq. 13, plus the stated white-box threat model.

assumptions (3)
  • domain assumption S(x;Mpt) estimates the a priori membership probability of x and subtracting it from S(x;M) increases membership separability (Eq. 13).
    Invoked in Section IV-C for all five proposed attacks; no proof or condition is given, and it fails if Mpt already encodes the fine-tuning distribution.
  • domain assumption The adversary knows the exact pre-trained model Mpt and has white-box access to Mft and Mpt (Section IV-A).
    The full attack pipeline depends on this access; the authors explicitly exclude closed-source and API-only deployments.
  • domain assumption Non-members are sampled from the same underlying distribution as members but disjoint from the fine-tuning and validation sets (Section V-A).
    This makes AUC a measure of membership signal, but it assumes the attacker's candidate pool matches the training distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models." pith.science (2026). https://pith.science/paper/YFBZ4OHU

@misc{pith2026250718302,
  author       = {Pith},
  title        = {Pith review of: LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YFBZ4OHU}},
  note         = {Machine review of arXiv:2507.18302}
}
read the original abstract

Language Models (LMs) typically adhere to a "pre-training and fine-tuning" paradigm, where a universal pre-trained model can be fine-tuned to cater to various specialized domains. Low-Rank Adaptation (LoRA) has gained the most widespread use in LM fine-tuning due to its lightweight computational cost and remarkable performance. Because the proportion of parameters tuned by LoRA is relatively small, there might be a misleading impression that the LoRA fine-tuning data is invulnerable to Membership Inference Attacks (MIAs). However, we identify that utilizing the pre-trained model can induce more information leakage, which is neglected by existing MIAs. Therefore, we introduce LoRA-Leak, a holistic evaluation framework for MIAs against the fine-tuning datasets of LMs. LoRA-Leak incorporates fifteen membership inference attacks, including ten existing MIAs, and five improved MIAs that leverage the pre-trained model as a reference. In experiments, we apply LoRA-Leak to three advanced LMs across three popular natural language processing tasks, demonstrating that LoRA-based fine-tuned LMs are still vulnerable to MIAs (e.g., 0.775 AUC under conservative fine-tuning settings). We also applied LoRA-Leak to different fine-tuning settings to understand the resulting privacy risks. We further explore four defenses and find that only dropout and excluding specific LM layers during fine-tuning effectively mitigate MIA risks while maintaining utility. We highlight that under the "pre-training and fine-tuning" paradigm, the existence of the pre-trained model makes MIA a more severe risk for LoRA-based LMs. We hope that our findings can provide guidance on data privacy protection for specialized LM providers.

Figures

Figures reproduced from arXiv: 2507.18302 by the authors.

Figure 1
Figure 1. Overview of LoRA-Leak. LoRA-Leak aggregates in￾formation from the specialized fine-tuned model and its pre￾trained model to launch more powerful MIAs against LMs. For example, codeLlama [5] and AstroLLaMA [6] are the fine￾tuned variants of Llama-2 [4] on programming domain and astronomy domain, respectively. The performance of the fine-tuned specialized LMs hinges on two key factors: the fine-tuning algorithms and t… view at source ↗
Figure 2
Figure 2. The pipeline of LoRA-Leak. The sample x is fed to the target model to infer its membership. During the forward and back propagation, the internal states of the model can be perturbed as the dotted line indicates. Some internal states of the model can be extracted to initiate attacks as the red arrow indicates. These signals can be further calibrated by pre-trained models to obtain more effective MIAs. TABLE I: The l… view at source ↗
Figure 3
Figure 3. The perplexity of training and validation dataset as the fine-tuning epoch increases, as well as the best AUC achieved [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: The AUC achieved for three models trained on AG News, using different reference models to enhance the non-referenced [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The best AUC scores for non-referenced and pt [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: The AUC of various MIAs when fine-tuning the pre-trained models with different epochs [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: The AUC of various MIAs against the fine-tuned Llama-2 with varying dropout rates ( [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: The AUC of various MIAs against the fine-tuned Llama-2 with varying weight decay rates ( [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: The AUC of various MIAs against the fine-tuned Llama-2 with varying privacy budgets ( [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: The AUC of various MIAs against the fine-tuned Llama-2 with and without the [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: The training and validation loss during Llama-2 LoRA fine-tuning across three datasets when applying differential [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 26 canonical work pages

  1. [1]

    Adapting large language models via reading comprehension,

    D. Cheng, S. Huang, and F. Wei, “Adapting large language models via reading comprehension,” in The Twelfth International Conference on Learning Representations , 2024. [Online]. Available: https://openreview.net/forum?id=y886UXPEZ0

    work page 2024

  2. [2]

    AstroLLaMA: Towards specialized foundation models in astronomy,

    T. D. Nguyen, Y .-S. Ting, I. Ciuca, C. O’Neill, Z.-C. Sun, M. Jabło ´nska, S. Kruk, E. Perkowski, J. Miller, J. J. J. Li, J. Peek, K. Iyer, T. Rozanski, P. Khetarpal, S. Zaman, D. Brodrick, S. J. Rodriguez Mendez, T. Bui, A. Goodman, A. Accomazzi, J. Naiman, J. Cranney, K. Schawinski, and R. Raileanu, “AstroLLaMA: Towards specialized foundation models in...

    work page 2023

  3. [3]

    Chatgpt,

    OpenAI, “Chatgpt,” https://openai.com/index/chatgpt/, 2022

    2022

  4. [4]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V . Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V . Kerkez, M. Khabsa, I. Kloumann, A. Koren...

    2023

  5. [5]

    Code llama: Open foundation models for code,

    B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y . Adi, J. Liu, T. Remez, J. Rapin et al., “Code llama: Open foundation models for code,” arXiv preprint arXiv:2308.12950 , 2023

    arXiv 2023

  6. [6]

    Astrollama: Towards specialized foundation models in astronomy,

    T. D. Nguyen, Y .-S. Ting, I. Ciuc˘a, C. O’Neill, Z.-C. Sun, M. Jabło ´nska, S. Kruk, E. Perkowski, J. Miller, J. Li, J. Peek, K. Iyer, T. R ´o˙za´nski, P. Khetarpal, S. Zaman, D. Brodrick, S. J. R. M ´endez, T. Bui, A. Goodman, A. Accomazzi, J. Naiman, J. Cranney, K. Schawinski, and UniverseTBD, “Astrollama: Towards specialized foundation models in astro...

    arXiv 2023

  7. [7]

    Llamafactory: Unified efficient fine-tuning of 100+ language models,

    Y . Zheng, R. Zhang, J. Zhang, Y . Ye, Z. Luo, Z. Feng, and Y . Ma, “Llamafactory: Unified efficient fine-tuning of 100+ language models,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) . Bangkok, Thailand: Association for Computational Linguistics, 2024. [Online]. Available: ht...

    arXiv 2024

  8. [8]

    Peft: State-of-the-art parameter-efficient fine-tuning methods,

    S. Mangrulkar, S. Gugger, L. Debut, Y . Belkada, S. Paul, and B. Bossan, “Peft: State-of-the-art parameter-efficient fine-tuning methods,” https:// github.com/huggingface/peft, 2022

    2022

Show all 58 references
  1. [9]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations , 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  2. [10]

    The philosopher’s stone: Trojaning plugins of large language models,

    T. Dong, M. Xue, G. Chen, R. Holland, Y . Meng, S. Li, Z. Liu, and H. Zhu, “The philosopher’s stone: Trojaning plugins of large language models,” 2024. [Online]. Available: https://arxiv.org/abs/2312.00374v2

    2024 arXiv

  3. [11]

    Qlora: Efficient finetuning of quantized llms,

    T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “Qlora: Efficient finetuning of quantized llms,”arXiv preprint arXiv:2305.14314, 2023

    2023 arXiv

  4. [12]

    Membership inference attacks from first principles,

    N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tram `er, “Membership inference attacks from first principles,” in 2022 IEEE Symposium on Security and Privacy (SP) , 2022, pp. 1897–1914

    2022

  5. [13]

    Last one standing: A comparative analysis of security and privacy of soft prompt tuning, lora, and in-context learning,

    R. Wen, T. Wang, M. Backes, Y . Zhang, and A. Salem, “Last one standing: A comparative analysis of security and privacy of soft prompt tuning, lora, and in-context learning,” 2023

    2023

  6. [14]

    Precurious: How innocent pre-trained language models turn into privacy traps,

    R. Liu, T. Wang, Y . Cao, and L. Xiong, “Precurious: How innocent pre-trained language models turn into privacy traps,” 2024

    2024

  7. [15]

    Character-level convolutional networks for text classification,

    X. Zhang, J. Zhao, and Y . LeCun, “Character-level convolutional networks for text classification,” in Advances in Neural Information Processing Systems , C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, Eds., vol. 28. Curran Associates, Inc., 2015. [Online]. Avail...

    2015

  8. [16]

    Openassistant conversations - democra- tizing large language model alignment,

    A. K ¨opf, Y . Kilcher, D. von R ¨utte, S. Anagnostidis, Z.-R. Tam, K. Stevens, A. Barhoum, N. M. Duc, O. Stanley, R. Nagyfi, S. ES, S. Suri, D. Glushkov, A. Dantuluri, A. Maguire, C. Schuhmann, H. Nguyen, and A. Mattick, “Openassistant conversations - democra- tizing large la...

    2024

  9. [17]

    What disease does this patient have? a large-scale open domain question answering dataset from medical exams,

    D. Jin, E. Pan, N. Oufattole, W.-H. Weng, H. Fang, and P. Szolovits, “What disease does this patient have? a large-scale open domain question answering dataset from medical exams,” arXiv preprint arXiv:2009.13081, 2020

    2009 arXiv

  10. [18]

    Membership inference attack susceptibility of clinical language models,

    A. Jagannatha, B. P. S. Rawat, and H. Yu, “Membership inference attack susceptibility of clinical language models,” 2021. [Online]. Available: https://arxiv.org/abs/2104.08305

    2021 arXiv

  11. [19]

    Quantifying privacy risks of masked language models using membership inference attacks,

    F. Mireshghallah, K. Goyal, A. Uniyal, T. Berg-Kirkpatrick, and R. Shokri, “Quantifying privacy risks of masked language models using membership inference attacks,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , Y . Goldberg, Z. Koz...

    2022

  12. [20]

    Extracting training data from large language models,

    N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-V oss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson et al., “Extracting training data from large language models,” in 30th USENIX Security Symposium (USENIX Security 21) , 2021, pp. 2633–2650

    2021

  13. [21]

    Membership inference attacks against language models via neighbourhood comparison,

    J. Mattern, F. Mireshghallah, Z. Jin, B. Schoelkopf, M. Sachan, and T. Berg-Kirkpatrick, “Membership inference attacks against language models via neighbourhood comparison,” in Findings of the Association for Computational Linguistics: ACL 2023 , A. Rogers, J. Boyd- Graber, an...

    2023

  14. [22]

    Membership inference attacks against fine-tuned large language models via self-prompt calibration,

    W. Fu, H. Wang, C. Gao, G. Liu, Y . Li, and T. Jiang, “Membership inference attacks against fine-tuned large language models via self-prompt calibration,” in Advances in Neural Information Processing Systems , A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak...

    2024

  15. [23]

    MoPe: Model perturbation based privacy attacks on language models,

    M. Li, J. Wang, J. Wang, and S. Neel, “MoPe: Model perturbation based privacy attacks on language models,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computationa...

    2023

  16. [24]

    Detecting pretraining data from large language models,

    W. Shi, A. Ajith, M. Xia, Y . Huang, D. Liu, T. Blevins, D. Chen, and L. Zettlemoyer, “Detecting pretraining data from large language models,” 2023

    2023

  17. [25]

    Min-k%++: Improved baseline for detecting pre-training data from large language models,

    J. Zhang, J. Sun, E. Yeats, Y . Ouyang, M. Kuo, J. Zhang, H. Yang, and H. Li, “Min-k%++: Improved baseline for detecting pre-training data from large language models,” arXiv preprint arXiv:2404.02936 , 2024

    2024 arXiv

  18. [26]

    Pandora’s white-box: Increased training data leakage in open llms,

    J. G. Wang, J. Wang, M. Li, and S. Neel, “Pandora’s white-box: Increased training data leakage in open llms,” 2024

    2024

  19. [27]

    Membership Infer- ence Attacks Against Machine Learning Models,

    R. Shokri, M. Stronati, C. Song, and V . Shmatikov, “Membership Infer- ence Attacks Against Machine Learning Models,” in IEEE Symposium on Security and Privacy (S&P) . IEEE, 2017, pp. 3–18

    2017

  20. [28]

    Comprehensive Privacy Anal- ysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning,

    M. Nasr, R. Shokri, and A. Houmansadr, “Comprehensive Privacy Anal- ysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning,” in IEEE Symposium on Security and Privacy (S&P) . IEEE, 2019, pp. 1021–1035

    2019

  21. [29]

    Stolen memories: Leveraging model memorization for calibrated White-Box membership inference,

    K. Leino and M. Fredrikson, “Stolen memories: Leveraging model memorization for calibrated White-Box membership inference,” in 29th USENIX Security Symposium (USENIX Security 20) . USENIX Association, Aug. 2020, pp. 1605–1622. [Online]. Available: https: //www.usenix.org/confe...

    2020

  22. [30]

    Privacy risks of securing machine learning models against adversarial examples,

    L. Song, R. Shokri, and P. Mittal, “Privacy risks of securing machine learning models against adversarial examples,” in Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’19. New York, NY , USA: Association for Computing Machinery,...

    2019

  23. [31]

    Privacy risk in machine learning: Analyzing the connection to overfitting,

    S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the connection to overfitting,” in 2018 IEEE 31st Computer Security Foundations Symposium (CSF) , 2018, pp. 268–282

    2018

  24. [32]

    Quantifying membership inference vulnerability via generalization gap and other model metrics,

    J. W. Bentley, D. Gibney, G. Hoppenworth, and S. K. Jha, “Quantifying membership inference vulnerability via generalization gap and other model metrics,” 2020. [Online]. Available: https: //arxiv.org/abs/2009.05669

    2020 arXiv

  25. [33]

    Sok: Membership inference attacks on llms are JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13 rushing nowhere (and how to fix it),

    M. Meeus, I. Shilov, S. Jain, M. Faysse, M. Rei, and Y .-A. de Montjoye, “Sok: Membership inference attacks on llms are JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13 rushing nowhere (and how to fix it),” 2024. [Online]. Available: https://arxiv.org/abs/2406.17975

    2021 arXiv

  26. [34]

    Blind baselines beat membership inference attacks for foundation models,

    D. Das, J. Zhang, and F. Tram `er, “Blind baselines beat membership inference attacks for foundation models,” 2024. [Online]. Available: https://arxiv.org/abs/2406.16201

    2024 arXiv

  27. [35]

    Do membership inference attacks work on large language models?

    M. Duan, A. Suri, N. Mireshghallah, S. Min, W. Shi, L. Zettlemoyer, Y . Tsvetkov, Y . Choi, D. Evans, and H. Hajishirzi, “Do membership inference attacks work on large language models?” 2024. [Online]. Available: https://arxiv.org/abs/2402.07841

    2024 arXiv

  28. [36]

    Memorization in nlp fine-tuning methods,

    F. Mireshghallah, A. Uniyal, T. Wang, D. Evans, and T. Berg- Kirkpatrick, “Memorization in nlp fine-tuning methods,” 2022. [Online]. Available: https://arxiv.org/abs/2205.12506

    2022 arXiv

  29. [37]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” 2019. [Online]. Available: https://arxiv.org/abs/1810.04805

    2019 arXiv

  30. [38]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research, vol. 21, no. 140, pp. 1–67, 2020. [Online]. Available: ht...

    2020

  31. [39]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  32. [40]

    Pythia: A suite for analyzing large language models across training and scaling,

    S. Biderman, H. Schoelkopf, Q. G. Anthony, H. Bradley, K. O’Brien, E. Hallahan, M. A. Khan, S. Purohit, U. S. Prashanth, E. Raff et al. , “Pythia: A suite for analyzing large language models across training and scaling,” in International Conference on Machine Learning . PMLR, ...

    2023

  33. [41]

    Openassistant top-1 conversation threads,

    OpenAssistant, “Openassistant top-1 conversation threads,” 2023. [Online]. Available: https://huggingface.co/datasets/OpenAssistant/ oasst top1 2023-08-25

    2023

  34. [42]

    Chat markup language,

    OpenAI, “Chat markup language,” https://github.com/openai/ openai-python/blob/284c1799070c723c6a553337134148a7ab088dd8/ chatml.md, 2020

    2020

  35. [43]

    Tl;dr news dataset,

    J. Belveze, “Tl;dr news dataset,” 2023. [Online]. Available: https: //huggingface.co/datasets/JulesBelveze/tldr news

    2023

  36. [44]

    Dropout: A simple way to prevent neural networks from overfitting,

    N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research , vol. 15, no. 56, pp. 1929–1958, 2014. [Online]. Available: http://jmlr.org/papers/v15/srivas...

    1929

  37. [45]

    A simple weight decay can improve gener- alization,

    A. Krogh and J. Hertz, “A simple weight decay can improve gener- alization,” Advances in neural information processing systems , vol. 4, 1991

    1991

  38. [46]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations , 2019. [Online]. Available: https://openreview.net/forum?id=Bkg6RiCqY7

    2019

  39. [47]

    On the effectiveness of regulariza- tion against membership inference attacks,

    Y . Kaya, S. Hong, and T. Dumitras, “On the effectiveness of regulariza- tion against membership inference attacks,” 2020

    2020

  40. [48]

    Calibrating noise to sensitivity in private data analysis,

    C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” inTheory of Cryptography, S. Halevi and T. Rabin, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 265–284

    2006

  41. [49]

    Differentially private fine-tuning of language models,

    D. Yu, S. Naik, A. Backurs, S. Gopi, H. A. Inan, G. Kamath, J. Kulkarni, Y . T. Lee, A. Manoel, L. Wutschitz, S. Yekhanin, and H. Zhang, “Differentially private fine-tuning of language models,” in International Conference on Learning Representations , 2022. [Online]. Available...

    2022

  42. [50]

    dp-transformers: Training transformer models with differential privacy,

    L. Wutschitz, H. A. Inan, and A. Manoel, “dp-transformers: Training transformer models with differential privacy,” https://www.microsoft. com/en-us/research/project/dp-transformers, August 2022

    2022

  43. [51]

    Openai api reference,

    OpenAI, “Openai api reference,” 2024, accessed: 2024-09-05. [Online]. Available: https://platform.openai.com/docs/api-reference/chat

    2024

  44. [52]

    Hugging face api inference documentation,

    H. Face, “Hugging face api inference documentation,” 2024, accessed: 2024-09-05. [Online]. Available: https://huggingface.co/ docs/api-inference/index

    2024

  45. [53]

    Label- Only Membership Inference Attacks,

    C. A. C. Choo, F. Tram `er, N. Carlini, and N. Papernot, “Label- Only Membership Inference Attacks,” in International Conference on Machine Learning (ICML) . PMLR, 2021, pp. 1964–1974

    2021

  46. [54]

    Membership Leakage in Label-Only Exposures,

    Z. Li and Y . Zhang, “Membership Leakage in Label-Only Exposures,” in ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM, 2021, pp. 880–895

    2021

  47. [55]

    Trl: Transformer reinforcement learning,

    L. von Werra, Y . Belkada, L. Tunstall, E. Beeching et al. , “Trl: Transformer reinforcement learning,” https://github.com/huggingface/trl, 2020

    2020

  48. [56]

    Dora: Weight-decomposed low-rank adaptation,

    S.-Y . Liu, C.-Y . Wang, H. Yin, P. Molchanov et al. , “Dora: Weight-decomposed low-rank adaptation,” 2024. [Online]. Available: https://arxiv.org/abs/2402.09353

    2024 arXiv

  49. [57]

    The power of scale for parameter-efficient prompt tuning,

    B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih, Eds. Online and Punta Cana, Domin...

    2021

  50. [58]

    Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,

    H. Liu, D. Tam, M. Muqeeth, J. Mohta et al. , “Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,” in Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave et al. , Eds., vol. 35. Curran Associa...

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.