Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

This paper claims that the decisive signal for membership inference in fine-tuned LLMs is the 'optimization gap' — the remaining loss-reduction potential of a sample — and that a training-free in-context probing technique can estimate this

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 15:34 UTC pith:76OAKKLU

load-bearing objection Solid empirical MIA paper; the optimization-gap framing is validated only for the reference variant, and the headline self-perturbation attack works but is better described as a perturbation heuristic. the 4 major comments →

arxiv 2512.16292 v2 pith:76OAKKLU submitted 2025-12-18 cs.CR cs.LG

In-Context Probing for Membership Inference in Fine-Tuned Language Models

Zhexi Lu , Hongliang Chi , Nathalie Baracaldo , Swanand Ravindra Kadhe , Yuseok Jeon , Lei Yu This is my paper
classification cs.CR cs.LG
keywords membership inferenceoptimization gapin-context learningin-context probingfine-tuned LLMsblack-box attackprivacy auditingdiminishing returns
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the signal separating members from non-members in a fine-tuned LLM is the 'optimization gap': how much loss could still be reduced if training continued on that sample. Members have already been optimized, so their gap is near zero; non-members show large potential gains. The paper proposes estimating this gap in a black-box setting by in-context probing — prepending carefully chosen contexts and measuring the log-likelihood improvement. This yields a membership score that outperforms previous black-box attacks, especially with a self-perturbation strategy that requires no reference data. If correct, it provides a principled, training-free way to audit privacy exposure in deployed models.

Core claim

The discovery is that the optimization gap is a fundamental and usable membership signal: at convergence, member samples have little remaining loss-reduction potential, while non-members retain substantial potential (measured per-sample: non-members mean loss reduction 0.368 vs members 0.125). The paper shows that in-context probing can approximate this gap without any parameter updates, and that the ICP score — the log-likelihood change induced by a probe context — correlates with the true single-step gradient loss reduction (Spearman 0.547). This grounding distinguishes the method from earlier heuristic perturbations.

What carries the argument

The central object is the Optimization Gap, defined as the difference between the loss at a converged model and the loss after one additional optimization step on that sample. The machinery that carries the argument is In-Context Probing (ICP): a training-free, black-box proxy that prepends a probe context to the input and computes the log-likelihood change; the minimum score over multiple probes is the membership signal.

Load-bearing premise

The entire method assumes that in-context probing faithfully mimics a real fine-tuning step; the only direct evidence is a moderate correlation (0.547), and when that approximation degrades (as with Pythia-2.8B) the attack's low-FPR performance collapses.

What would settle it

Compute true per-sample loss reductions from an actual fine-tuning step on a holdout set and compare them against ICP scores across a broad model family; if the Spearman correlation falls below some floor, or if a model with strong ICL still shows no gap, the claim fails. Concretely, the paper itself notes Pythia-2.8B with CNN-DM yields TPR@1%FPR 0.144 — a scenario where the proxy breaks.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Fine-tuned models' membership exposure can be measured with only query access and no reference models, using the self-perturbation variant.
  • The attack excels at high-precision settings: for example, TPR@1%FPR reaches 0.518 on CNN-DM, far above prior reference-free attacks.
  • Instruction-tuned models are more vulnerable than their base versions, because stronger in-context learning makes the probing approximation more faithful.
  • PEFT setups (LoRA, QLoRA) remain vulnerable; larger adapter rank increases memorization and attack success, while 4-bit quantization mitigates it.
  • Differential privacy (DP-SGD) suppresses all attacks including this one, but ICP-MIA still leads among reference-free methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The moderate correlation (0.547) suggests the grounding is partial; a more faithful proxy might come from multi-step in-context updates or looped inference, which could sharpen the signal at low FPR.
  • The mask-rate ablation's single-peaked curve implies the probe must remove exactly the right amount of information; this could be used as a principled design rule for future perturbation attacks.
  • The recency effect found under fixed learning rates (later training partitions more vulnerable) implies that fine-tuning order and schedule are not neutral: a deployed model's privacy risk may be unevenly distributed across its training data, which auditing tools should report per stratum.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes ICP-MIA, a black-box membership inference attack against fine-tuned LLMs. The authors introduce the "Optimization Gap"—the remaining loss-reduction potential of a sample—as a membership signal, and propose to estimate it via in-context probing: prepending a probe context and measuring the change in log-likelihood of the target response. Two probe families are proposed: reference-data-based (semantically retrieved examples from a public dataset) and self-perturbation (randomly masked or LLM-generated variants). The paper reports that, on three datasets and three target models, ICP-MIA-SP outperforms reference-free baselines (e.g., AUC 0.942 vs 0.847 for ReCaLL on HealthCareMagic with Llama-3.2-3B-Instruct) and is competitive with reference-model attacks, while requiring no training. The paper also includes ablations for mask rate, K, generator choice, PEFT methods, and DP-SGD.

Significance. If the empirical results are robust, the proposed attack is practically useful: it is reference-free (SP variant), requires only logit access, and improves TPR at low FPR, which matters for real-world auditing. The underlying hypothesis—that membership leaves a trace in residual optimization potential—is interesting and the control experiment (Fig. 3) provides direct support. The paper also provides a detailed artifact and a thorough ablation suite. However, the central theoretical claim suffers from a validation gap between the proxy used to test the optimization-gap link and the probes used in the actual attack, and the reported gains come from point estimates with hyperparameters selected on the same benchmarks. These issues undercut the claim of a 'theoretically grounded' signal for the SP variant, but the attack itself may still be a strong heuristic.

major comments (4)
  1. [Section IV-C vs. V-B.2, VI-C] The only empirical evidence that ICP scores approximate true gradient-based loss reduction (Fig. 4, Spearman ρ=0.547) is obtained with reference probes from the aligned iCliniq dataset, selecting the top-20 similar sample that maximizes the target's conditional likelihood. The actual attack variants—ICP-MIA-SP with random masking/generation, and ICP-MIA-Ref with Dolly-15k—use different probe distributions. No experiment connects random-mask or generated probes to a true fine-tuning step. Indeed, §VI-C explains the success of random masking by "the probability of finding an effective probe", which is a perturbation-search explanation, not an optimization-gap one. Thus the central novelty claim ("we introduce the Optimization Gap as a fundamental signal" and "ICP-MIA is not another perturbation heuristic") is unsupported for the attacks that produce the main results. Please either (a) vali
  2. [Section VI-C, Fig. 7; Section VI-G, Fig. 11] The hyperparameters of the attack—mask ratio p and number of probes K—are selected using the same evaluation datasets. Fig. 7 shows a non-monotonic sensitivity to p (optimal around 30-70%), and Fig. 11 shows gains from increasing K; the reported default p=0.7 and K=5/10 are chosen with knowledge of the test results. This is a form of test-data leakage that can inflate the reported performance relative to baselines with fixed/standard hyperparameters. Please use a separate validation split for hyperparameter selection, or report results across a grid of p and K without claiming optimality. Also explain the choice K=5 for SP when Table VIII shows K=20 frequently improves AUC/TPR.
  3. [Table I] Every metric is reported once, with no variance across seeds, data splits, or bootstrap resampling. This is especially problematic for TPR@1%FPR, which is computed on 500 non-members (5 expected false positives at 1%); the estimate has a standard error of roughly ±0.02–0.05, comparable to several reported differences. The claim of "significantly outperforms" (Abstract, §I) is not statistically supported. Please provide repeated runs with different seeds and/or bootstrap confidence intervals for the key comparisons, and state the number of samples and thresholding procedure.
  4. [Section IV-A and IV-B] The "theoretical grounding" is overclaimed. The power-law decay of training loss is cited but never used in the derivation; the Optimization Gap is defined but no formal connection between it and the ICP score is established beyond the moderate Spearman correlation in Fig. 4. The empirical control (Fig. 3) is supportive but only for the gap itself. The paper should either (a) provide a mechanistic derivation or a more rigorous justification of why prepending a probe context induces the same loss-reduction potential as a real gradient step, or (b) lower the strength of the claim from "grounded in training dynamics" to "motivated by training dynamics." This issue is less about correctness than about calibration of claims, but it is load-bearing for the paper's framing.
minor comments (6)
  1. [Eq. (4) and (7)] The sign convention in Eq. (4) is confusing: ΔLL(s) = LL(y|x;M) − LL(y|x;M′) is a negative number when the fine-tuned model assigns higher likelihood, yet the text calls it "optimization gain." Eq. (7) defines ICP score similarly and later treats negative ICP scores as indicating larger gains. Please clarify the sign conventions so the relationship between ΔLL, the Optimization Gap, and the final membership score is unambiguous.
  2. [Section IV-C] The model names are inconsistent: "LLama-3.3-3B-instruct" appears in §IV-C, but the experimental models are LLaMA-3.2-3B and LLaMA-3.2-3B-Instruct (Table I). Please correct.
  3. [Section VI-A and Table II] Generator model names are inconsistent: §VI-A lists "Qwen2-72B-Instruct" while Table II uses "Qwen2.5-72B-Instruct." Please standardize.
  4. [Figure 9] The x-axis label "Number of Pretrain Data in Nonmembers" is ambiguous, and the axis values (100, 70, 50, 30) decrease from left to right while the text says performance degrades "as the proportion of pre-trained samples increases." Please clarify what the axis represents and the direction.
  5. [Artifact Appendix A] The appendix says to fine-tune for 3 epochs and use the 2nd epoch checkpoint, while §VI-A states "all models are fully fine-tuned for two epochs." Please reconcile this discrepancy.
  6. [Eq. (8)] The min over K probes in Eq. (8) is presented as a central design choice, but no justification is given for why the minimum (rather than the mean or the maximum over probes) is the correct aggregation. If this follows from the optimization-gap interpretation, state it; otherwise, provide an empirical comparison of aggregation methods.

Circularity Check

0 steps flagged

ICP-MIA is not circular: the Optimization-Gap signal is validated by a direct gradient-update control, the ICP score is parameter-free, and the only self-citation is non-load-bearing.

full rationale

The paper's derivation chain is: Optimization Gap (Sec. IV-A) -> hypothesis that in-context probes approximate a real fine-tuning step (Sec. IV-B) -> empirical validation against true single-step gradient loss reduction (Sec. IV-C, Fig. 4) -> attack score (Eq. 8). Each link is testable and none is defined in terms of the target label. The ICP score, Eq. (7), is simply LL(y|x;M) - LL(y|C+x;M) and contains no fitted parameters; Eq. (8) is a min over candidate probes, not a fit to membership labels. The Optimization-Gap hypothesis is supported by an independent control experiment (Fig. 3) in which true per-sample loss reductions are measured on known member and non-member samples. The correlation in Fig. 4 (Spearman 0.547) is moderate, but it is evidence rather than a tautology: the paper explicitly conditions the proxy link on 'If ICL indeed mimics gradient-based optimization' and then tests that condition. The skeptical concern that the validation used semantically aligned iCliniq probes while the headline ICP-MIA-SP attack uses random-mask/generation probes is an evidential gap or overclaim about generalization, not a circular reduction; the paper does not define the SP score in terms of the validation quantity, and it openly reports the Pythia-2.8B collapse (TPR@1%FPR 0.144) as an ICL-capability limitation. The only self-citation, reference [17], appears in a general list of MIA applications ('enable privacy auditing in deployed systems [15]-[17]') and is not load-bearing for the central claim. Hence no circular step meets the required quote-and-reduction standard.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 1 invented entities

The attack rests on the assumption that in-context probes simulate a fine-tuning step; this is empirically validated only by a moderate correlation. Hyperparameters (mask rate, K, retrieval top-k) are chosen using the same evaluation benchmarks, inflating reported performance. No new physical or mathematical entities are introduced beyond the conceptual 'Optimization Gap'.

free parameters (4)
  • mask_rate p = 0.7
    Selected as the peak of the non-monotonic ablation in Fig. 7 on the same datasets/models used for evaluation; performance in Table I depends on this choice.
  • K (number of candidate probes) = 5 (SP), 10 (Ref)
    Default chosen via top-K analysis (Fig. 11) on the evaluation benchmarks; TPR@1%FPR is sensitive to K.
  • retrieval top-k (for ICP-MIA-Ref) = 20 (validation)
    Used in the proxy validation (Sec. IV-C) and probe construction (Sec. V-B.1); chosen by the authors without a separate held-out tuning set.
  • mask perturbation count = 20
    Artifact generates 20 masked variants per sample (Appendix A); how this maps to K=5 in the paper is not fully specified.
axioms (4)
  • domain assumption LLM fine-tuning loss follows a power-law decay L(t) ≈ C t^{-α} + L∞ (Sec. IV-A)
    Cited from scaling-law literature [50]–[52]; used to motivate that members are near convergence. Not checked for the 2-epoch models used in the main experiments.
  • domain assumption In-context probing approximates a gradient step, so LL(y|C⊕x;M) ≈ LL(y|x;M′) (Sec. IV-B, Eqs. 4–7)
    Load-bearing: the entire ICP score is interpreted as an optimization-gap estimate. Only empirical support is Spearman 0.547 (Fig. 4), and the authors note the approximation fails for weaker ICL models (Pythia).
  • domain assumption Members and non-members are drawn from the same distribution, and target data is absent from pretraining (Sec. III-B)
    Needed for a fair MIA game; the residual-memorization experiments show the second part can be relaxed with only marginal loss.
  • ad hoc to paper Taking the minimum ICP score over K probes separates members from non-members (Eq. 8)
    Heuristic with no theoretical guarantee; supported only by experiments. The min operator effectively selects the best probe per sample, but does not fit labels.
invented entities (1)
  • Optimization Gap no independent evidence
    purpose: Conceptual signal of membership: remaining loss-reduction potential of a sample after training.
    The paper defines it in Sec. IV-A but never measures it directly; it is only proxied by ICP, and the proxy's correlation with a true gradient step is moderate (0.547). No external falsifiable prediction beyond attack performance.

pith-pipeline@v1.3.0-alltime-deepseek · 30374 in / 15661 out tokens · 134968 ms · 2026-08-03T15:34:08.414760+00:00 · methodology

0 comments
read the original abstract

Membership inference attacks (MIAs) pose a critical privacy threat to fine-tuned large language models (LLMs), especially when models are adapted to domain-specific tasks using sensitive data. While prior black-box MIA techniques rely on confidence scores or token likelihoods, these signals are often entangled with a sample's intrinsic properties - such as content difficulty or rarity - leading to poor generalization and low signal-to-noise ratios. In this paper, we propose ICP-MIA, a novel MIA framework grounded in the theory of training dynamics, particularly the phenomenon of diminishing returns during optimization. We introduce the Optimization Gap as a fundamental signal of membership: at convergence, member samples exhibit minimal remaining loss-reduction potential, while non-members retain significant potential for further optimization. To estimate this gap in a black-box setting, we propose In-Context Probing (ICP), a training-free method that simulates fine-tuning-like behavior via strategically constructed input contexts. We propose two probing strategies: reference-data-based (using semantically similar public samples) and self-perturbation (via masking or generation). Experiments on three tasks and multiple LLMs show that ICP-MIA significantly outperforms prior black-box MIAs, particularly at low false positive rates. We further analyze how reference data alignment, model type, PEFT configurations, and training schedules affect attack effectiveness. Our findings establish ICP-MIA as a practical and theoretically grounded framework for auditing privacy risks in deployed LLMs.

Figures

Figures reproduced from arXiv: 2512.16292 by Hongliang Chi, Lei Yu, Nathalie Baracaldo, Swanand Ravindra Kadhe, Yuseok Jeon, Zhexi Lu.

Figure 1
Figure 1. Figure 1: Log-likelihood improvement distribution on the Health [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Empirical illustration of diminishing returns during [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Fine-tuning with Members V.S. Non-Members [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Correlation between actual single-step training loss [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Impact of reference data (prefix pool) and model type on the fidelity of the ICP proxy, measured by Spearman correlation. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of Different Masking-based Probing Strategies [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Mask-rate ablation for the masking-based ICP-MIA-SP variant. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: ICP scores distribution under different masking per [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The Impact of Residual Memorization of Non-Members [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: The Impact of Training Sequence 17 [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Efficiency analysis on Top K 0 1 2 3 4 5 6 7 8 9 Partition Index 0.5 0.6 0.7 0.8 0.9 1.0 AUC 0.536 0.638 0.704 0.748 0.816 0.516 0.600 0.629 0.685 0.724 3 Epochs (1st) 0 1 2 3 4 5 6 7 8 9 Partition Index 0.5 0.6 0.7 0.8 0.9 1.0 AUC 0.842 0.874 0.902 0.914 0.937 0.786 0.853 0.882 0.909 0.925 3 Epochs (2nd) 0 1 2 3 4 5 6 7 8 9 Partition Index 0.5 0.6 0.7 0.8 0.9 1.0 AUC 0.936 0.935 0.936 0.936 0.935 0.951 0… view at source ↗
Figure 12
Figure 12. Figure 12: Training Sequence and Learning Rate Schedule Lead to Different MIA Vulnerability [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Ablation study on temperature in Prefix Generation [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Probing Memorization of Tabular In-Context Learning

    cs.LG 2026-06 unverdicted novelty 7.0

    A new probing framework detects moderate parametric memorization signals in tabular in-context learning models under single-task fine-tuning, strongest on low-cardinality tasks, but signals largely disappear under rea...

Reference graph

Works this paper leans on

86 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosaleet al., “Llama 2: Open foundation and fine-tuned chat models,”arXiv preprint arXiv:2307.09288, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Recent advances in natural language processing via large pre-trained language models: A survey,

    B. Min, H. Ross, E. Sulem, A. P. B. Veyseh, T. H. Nguyen, O. Sainz, E. Agirre, I. Heintz, and D. Roth, “Recent advances in natural language processing via large pre-trained language models: A survey,”ACM Computing Surveys, vol. 56, no. 2, pp. 1–40, 2023

    2023

  3. [3]

    Fine-tuning methods for large language models in clinical medicine by supervised fine-tuning and direct preference optimization: Comparative evaluation,

    T. Savage, S. P. Ma, A. Boukil, E. Rangan, V . Patel, I. Lopez, and J. Chen, “Fine-tuning methods for large language models in clinical medicine by supervised fine-tuning and direct preference optimization: Comparative evaluation,”J Med Internet Res, vol. 27, no. e76048, p. e76048, 2025

    2025

  4. [4]

    Extracting training data from large language models,

    N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-V oss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingssonet al., “Extracting training data from large language models,” in30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 2633–2650

    2021

  5. [5]

    Detecting pretraining data from large language models,

    W. Shi, A. Ajith, M. Xia, Y . Huang, D. Liu, T. Blevins, D. Chen, and L. Zettlemoyer, “Detecting pretraining data from large language models,” inThe Twelfth International Conference on Learning Repre- sentations, 2023

    2023

  6. [6]

    Detecting training data of large language models via expectation maximization,

    G. Kim, Y . Li, E. Spiliopoulou, J. Ma, M. Ballesteros, and W. Y . Wang, “Detecting training data of large language models via expectation maximization,”arXiv preprint arXiv:2410.07582, 2024

    arXiv 2024

  7. [7]

    Differentially private fine- tuning of language models,

    D. Yu, S. Naik, A. Backurs, S. Gopi, H. A. Inan, G. Kamath, J. Kulkarni, Y . T. Lee, A. Manoel, L. Wutschitzet al., “Differentially private fine- tuning of language models,”arXiv preprint arXiv:2110.06500, 2021

    Pith/arXiv arXiv 2021

  8. [8]

    Membership inference attacks against machine learning models,

    R. Shokri, M. Stronati, C. Song, and V . Shmatikov, “Membership inference attacks against machine learning models,” in2017 IEEE symposium on security and privacy (SP). IEEE, 2017, pp. 3–18

    2017

  9. [9]

    Gan-leaks: A taxonomy of membership inference attacks against generative models,

    D. Chen, N. Yu, Y . Zhang, and M. Fritz, “Gan-leaks: A taxonomy of membership inference attacks against generative models,” inPro- ceedings of the 2020 ACM SIGSAC conference on computer and communications security, 2020, pp. 343–362

    2020

  10. [10]

    Please tell me more: Privacy impact of explainability through the lens of membership inference attack,

    H. Liu, Y . Wu, Z. Yu, and N. Zhang, “Please tell me more: Privacy impact of explainability through the lens of membership inference attack,” in2024 IEEE Symposium on Security and Privacy (SP). IEEE Computer Society, 2024, pp. 120–120

    2024

  11. [11]

    Membership inference attacks against diffusion models,

    T. Matsumoto, T. Miura, and N. Yanai, “Membership inference attacks against diffusion models,” in2023 IEEE Security and Privacy Workshops (SPW). IEEE, 2023, pp. 77–83

    2023

  12. [12]

    Lds-fl: Loss differential strategy based federated learning for privacy preserving,

    T. Wang, Q. Yang, K. Zhu, J. Wang, C. Su, and K. Sato, “Lds-fl: Loss differential strategy based federated learning for privacy preserving,” IEEE Transactions on Information Forensics and Security, 2023

    2023

  13. [13]

    Accuracy-privacy trade-off in deep ensemble: A membership inference perspective,

    S. Rezaei, Z. Shafiq, and X. Liu, “Accuracy-privacy trade-off in deep ensemble: A membership inference perspective,” in2023 IEEE Sympo- sium on Security and Privacy (SP). IEEE, 2023, pp. 364–381

    2023

  14. [14]

    Towards unbounded machine unlearning,

    M. Kurmanji, P. Triantafillou, J. Hayes, and E. Triantafillou, “Towards unbounded machine unlearning,”Advances in neural information pro- cessing systems, vol. 36, pp. 1957–1987, 2023

    1957

  15. [15]

    Quantifying privacy risks of masked language models using membership inference attacks,

    F. Mireshghallah, K. Goyal, A. Uniyal, T. Berg-Kirkpatrick, and R. Shokri, “Quantifying privacy risks of masked language models using membership inference attacks,”arXiv preprint arXiv:2203.03929, 2022

    Pith/arXiv arXiv 2022

  16. [16]

    Panoramia: Privacy auditing of machine learning models without retraining,

    M. Kazmi, H. Lautraite, A. Akbari, M. Soroco, Q. Tang, T. Wang, S. Gambs, and M. L ´ecuyer, “Panoramia: Privacy auditing of machine learning models without retraining,”arXiv preprint arXiv:2402.09477, 2024

    Pith/arXiv arXiv 2024

  17. [17]

    Membership inference attacks as privacy tools: Reliability, disparity and ensemble,

    Z. Wang, C. Zhang, Y . Chen, N. Baracaldo, S. R. Kadhe, and L. Yu, “Membership inference attacks as privacy tools: Reliability, disparity and ensemble,” inProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’25. Association for Computing Machinery, 2025, p. 1724–1738

    2025

  18. [18]

    An empirical analysis of memorization in fine-tuned au- toregressive language models,

    F. Mireshghallah, A. Uniyal, T. Wang, D. K. Evans, and T. Berg- Kirkpatrick, “An empirical analysis of memorization in fine-tuned au- toregressive language models,” inProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 1816– 1826

    2022

  19. [19]

    Do membership inference attacks work on large language models?

    M. Duan, A. Suri, N. Mireshghallah, S. Min, W. Shi, L. Zettlemoyer, Y . Tsvetkov, Y . Choi, D. Evans, and H. Hajishirzi, “Do membership inference attacks work on large language models?” inFirst Conference on Language Modeling, 2024

    2024

  20. [20]

    Membership inference attacks from first principles,

    N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer, “Membership inference attacks from first principles,” in2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022, pp. 1897–1914

    2022

  21. [21]

    Min-k%++: Improved baseline for pre-training data detection from large language models,

    J. Zhang, J. Sun, E. Yeats, Y . Ouyang, M. Kuo, J. Zhang, H. F. Yang, and H. Li, “Min-k%++: Improved baseline for pre-training data detection from large language models,” inThe Thirteenth International Conference on Learning Representations, 2025

    2025

  22. [22]

    Recall: Membership inference via relative conditional log-likelihoods,

    R. Xie, J. Wang, R. Huang, M. Zhang, R. Ge, J. Pei, N. Gong, and B. Dhingra, “Recall: Membership inference via relative conditional log-likelihoods,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 8671–8689

    2024

  23. [23]

    On the stability of fine-tuning bert: Misconceptions, explanations, and strong baselines,

    M. Mosbach, M. Andriushchenko, and D. Klakow, “On the stability of fine-tuning bert: Misconceptions, explanations, and strong baselines,” in International Conference on Learning Representations, 2020

    2020

  24. [24]

    Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves

    T. Domhan, J. T. Springenberg, and F. Hutter, “Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves.” inIJCAI, vol. 15, 2015, pp. 3460–8

    2015

  25. [25]

    The shape of learning curves: a review,

    T. Viering and M. Loog, “The shape of learning curves: a review,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 6, pp. 7799–7819, 2022

    2022

  26. [26]

    Membership inference attacks against fine-tuned large language models via self- prompt calibration,

    W. Fu, H. Wang, C. Gao, G. Liu, Y . Li, and T. Jiang, “Membership inference attacks against fine-tuned large language models via self- prompt calibration,” inThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  27. [27]

    Df-mia: A distribution-free membership inference attack on fine-tuned large language models,

    Z. Huang, Y . Liu, D. He, and Y . Li, “Df-mia: A distribution-free membership inference attack on fine-tuned large language models,” in 14 Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 1, 2025, pp. 343–351

    2025

  28. [28]

    Privacy risk in machine learning: Analyzing the connection to overfitting,

    S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the connection to overfitting,” in2018 IEEE 31st computer security foundations symposium (CSF). IEEE, 2018, pp. 268–282

    2018

  29. [29]

    Pretraining data detection for large language models: A divergence- based calibration method,

    W. Zhang, R. Zhang, J. Guo, M. de Rijke, Y . Fan, and X. Cheng, “Pretraining data detection for large language models: A divergence- based calibration method,” inEMNLP, 2024

    2024

  30. [30]

    Membership inference attacks against language models via neighbourhood comparison,

    J. Mattern, F. Mireshghallah, Z. Jin, B. Sch ¨olkopf, M. Sachan, and T. Berg-Kirkpatrick, “Membership inference attacks against language models via neighbourhood comparison,”arXiv preprint arXiv:2305.18462, 2023

    Pith/arXiv arXiv 2023

  31. [31]

    Con- recall: Detecting pre-training data in llms via contrastive decoding,

    C. Wang, Y . Wang, B. Hooi, Y . Cai, N. Peng, and K.-W. Chang, “Con- recall: Detecting pre-training data in llms via contrastive decoding,” inProceedings of the 31st International Conference on Computational Linguistics, 2025, pp. 1013–1026

    2025

  32. [32]

    Membership inference attacks against large vision-language models,

    Z. Li, Y . Wu, Y . Chen, F. Tonin, E. A. Rocamora, and V . Cevher, “Membership inference attacks against large vision-language models,” in Proceedings of the 38th International Conference on Neural Information Processing Systems, 2024, pp. 98 645–98 674

    2024

  33. [33]

    Black-box membership inference attacks against fine-tuned diffusion models,

    Y . Pang and T. Wang, “Black-box membership inference attacks against fine-tuned diffusion models,” in32nd Annual Network and Distributed System Security Symposium (NDSS) 2025, San Diego, California, USA, February 24–28, 2025. The Internet Society, 2025

    2025

  34. [34]

    Why can gpt learn in-context? language models secretly perform gradient descent as meta-optimizers,

    D. Dai, Y . Sun, L. Dong, Y . Hao, S. Ma, Z. Sui, and F. Wei, “Why can gpt learn in-context? language models secretly perform gradient descent as meta-optimizers,” inFindings of the Association for Computational Linguistics: ACL 2023, pp. 4005–4019

    2023

  35. [35]

    What learning algorithm is in-context learning? investigations with linear models,

    E. Aky ¨urek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou, “What learning algorithm is in-context learning? investigations with linear models,” inThe Eleventh International Conference on Learning Rep- resentations, 2022

    2022

  36. [36]

    Bypassing the expo- nential dependency: Looped transformers efficiently learn in-context by multi-step gradient descent,

    B. Chen, X. Li, Y . Liang, Z. Shi, and Z. Song, “Bypassing the expo- nential dependency: Looped transformers efficiently learn in-context by multi-step gradient descent,” inThe 28th International Conference on Artificial Intelligence and Statistics, 2024

    2024

  37. [37]

    Detail: Task demonstration attribution for interpretable in-context learning,

    Z. Zhou, X. Lin, X. Xu, A. Prakash, D. Rus, and B. K. H. Low, “Detail: Task demonstration attribution for interpretable in-context learning,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  38. [38]

    In-context probing: Toward building robust classifiers via probing large language models,

    A. Amini and M. Ciaramita, “In-context probing: Toward building robust classifiers via probing large language models,”arXiv preprint arXiv:2305.14171, 2023

    Pith/arXiv arXiv 2023

  39. [39]

    On the feasibility of in-context probing for data attribution,

    C. Jiao, W. Gao, A. Raghunathan, and C. Xiong, “On the feasibility of in-context probing for data attribution,” inFindings of the Associ- ation for Computational Linguistics: NAACL 2025. Association for Computational Linguistics, Apr. 2025, pp. 5140–5155

    2025

  40. [40]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models,” inInter- national Conference on Learning Representations, 2022

    2022

  41. [41]

    Dora: Weight-decomposed low-rank adap- tation,

    S.-Y . Liu, C.-Y . Wang, H. Yin, P. Molchanov, Y .-C. F. Wang, K.-T. Cheng, and M.-H. Chen, “Dora: Weight-decomposed low-rank adap- tation,” inForty-first International Conference on Machine Learning, 2024

    2024

  42. [42]

    Lora+: Efficient low rank adaptation of large models,

    S. Hayou, N. Ghosh, and B. Yu, “Lora+: Efficient low rank adaptation of large models,”arXiv preprint arXiv:2402.12354, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    The power of scale for parameter-efficient prompt tuning,

    B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” inProceedings of the 2021 Con- ference on Empirical Methods in Natural Language Processing. As- sociation for Computational Linguistics, 2021, pp. 3045–3059

    2021

  44. [44]

    Prefix-tuning: Optimizing continuous prompts for generation,

    X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,”arXiv preprint arXiv:2101.00190, 2021

    Pith/arXiv arXiv 2021

  45. [45]

    P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,

    X. Liu, K. Ji, Y . Fu, W. L. Tam, Z. Du, Z. Yang, and J. Tang, “P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,” 2022

    2022

  46. [46]

    Qlora: Efficient finetuning of quantized llms,

    T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “Qlora: Efficient finetuning of quantized llms,”Advances in neural information processing systems, vol. 36, pp. 10 088–10 115, 2023

    2023

  47. [47]

    Instruction fine-tuning: Does prompt loss matter?

    M. Huerta-Enochian and S. Y . Ko, “Instruction fine-tuning: Does prompt loss matter?” inProceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing. Association for Computational Linguistics, Nov. 2024, pp. 22 771–22 795

    2024

  48. [48]

    Efficient memory management for large language model serving with pagedattention,

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,” inProceedings of the 29th symposium on operating systems principles, 2023, pp. 611–626

    2023

  49. [49]

    Transformers: State- of-the-art natural language processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowiczet al., “Transformers: State- of-the-art natural language processing,” inProceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38–45

    2020

  50. [50]

    A dynamical model of neural scaling laws,

    B. Bordelon, A. B. Atanasov, and C. Pehlevan, “A dynamical model of neural scaling laws,” inForty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, 2024

    2024

  51. [51]

    Training compute-optimal large language models,

    J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark et al., “Training compute-optimal large language models,”arXiv preprint arXiv:2203.15556, 2022

    Pith/arXiv arXiv 2022

  52. [52]

    Scaling laws for neural language models,

    J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,”arXiv preprint arXiv:2001.08361, 2020

    Pith/arXiv arXiv 2001

  53. [53]

    One epoch is all you need,

    A. Komatsuzaki, “One epoch is all you need,”arXiv preprint arXiv:1906.06669, 2019

    Pith/arXiv arXiv 1906

  54. [54]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” inNAACL, 2019, pp. 4171–4186

    2019

  55. [55]

    Fine-tuning pretrained language models: Weight initializa- tions, data orders, and early stopping,

    J. Dodge, G. Ilharco, R. Schwartz, A. Farhadi, H. Hajishirzi, and N. Smith, “Fine-tuning pretrained language models: Weight initializa- tions, data orders, and early stopping,”arXiv preprint arXiv:2002.06305, 2020

    Pith/arXiv arXiv 2002

  56. [56]

    To repeat or not to repeat: Insights from scaling llm under token-crisis,

    F. Xue, Y . Fu, W. Zhou, Z. Zheng, and Y . You, “To repeat or not to repeat: Insights from scaling llm under token-crisis,”Advances in Neural Information Processing Systems, vol. 36, pp. 59 304–59 322, 2023

    2023

  57. [57]

    Pythia: A suite for analyzing large language models across training and scaling,

    S. Biderman, H. Schoelkopf, Q. G. Anthonyet al., “Pythia: A suite for analyzing large language models across training and scaling,” in International Conference on Machine Learning. PMLR, 2023, pp. 2397–2430

    2023

  58. [58]

    The llama 3 herd of models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughanet al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  59. [59]

    Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge,

    Y . Li, Z. Li, K. Zhang, R. Dan, S. Jiang, and Y . Zhang, “Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge,”Cureus, vol. 15, no. 6, 2023

    2023

  60. [60]

    Teaching machines to read and comprehend,

    K. M. Hermann, T. Kocisky, E. Grefenstette, L. Espeholt, W. Kay, M. Su- leyman, and P. Blunsom, “Teaching machines to read and comprehend,” Advances in neural information processing systems, vol. 28, 2015

    2015

  61. [61]

    Alpacare: Instruction-tuned large language models for medical application,

    X. Zhang, C. Tian, X. Yang, L. Chen, Z. Li, and L. R. Petzold, “Alpacare: Instruction-tuned large language models for medical application,”arXiv preprint arXiv:2310.14558, 2023

    Pith/arXiv arXiv 2023

  62. [62]

    Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization,

    S. Narayan, S. B. Cohen, and M. Lapata, “Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization,” inProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computa- tional Linguistics, Oct.-Nov. 2018, pp. 1797–1807

    2018

  63. [63]

    Character-level convolutional networks for text classification,

    X. Zhang, J. Zhao, and Y . LeCun, “Character-level convolutional networks for text classification,” inAdvances in Neural Information Processing Systems 28. Curran Associates, Inc., 2015, pp. 649–657

    2015

  64. [64]

    Sok: Membership inference attacks on llms are rushing nowhere (and how to fix it),

    M. Meeus, I. Shilov, S. Jain, M. Faysse, M. Rei, and Y .-A. de Montjoye, “Sok: Membership inference attacks on llms are rushing nowhere (and how to fix it),” in2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 2025, pp. 385–401

    2025

  65. [65]

    Qwen2.5 technical report,

    Qwen, A. Yang, B. Yang, B. Zhanget al., “Qwen2.5 technical report,” 2025

    2025

  66. [66]

    Mixtral of experts,

    A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bam- fordet al., “Mixtral of experts,”arXiv preprint arXiv:2401.04088, 2024

    Pith/arXiv arXiv 2024

  67. [67]

    Introducing gpt-4.1 in the api (incl. gpt-4.1 mini),

    OpenAI, “Introducing gpt-4.1 in the api (incl. gpt-4.1 mini),” OpenAI blog / documentation, 2025

    2025

  68. [68]

    Stanford alpaca: An instruction-following llama model,

    R. Taori, I. Gulrajani, T. Zhang, Y . Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Stanford alpaca: An instruction-following llama model,” 2023

    2023

  69. [69]

    TOFU: A task of fictitious unlearning for LLMs,

    P. Maini, Z. Feng, A. Schwarzschild, Z. C. Lipton, and J. Z. Kolter, “TOFU: A task of fictitious unlearning for LLMs,” inFirst Conference on Language Modeling, 2024

    2024

  70. [70]

    Dwork and A

    C. Dwork and A. Roth,The Algorithmic Foundations of Differential Privacy, ser. Foundations and Trends in Theoretical Computer Science. Now Publishers Inc., 2014, vol. 9, no. 3–4. 15

    2014

  71. [71]

    Deep learning with differential privacy,

    M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, 2016, pp. 308–318

    2016

  72. [72]

    Towards label-only membership inference attack against pre- trained large language models,

    Y . He, B. Li, L. Liu, Z. Ba, W. Dong, Y . Li, Z. Qin, K. Ren, and C. Chen, “Towards label-only membership inference attack against pre- trained large language models,” inUSENIX Security, 2025. APPENDIX A. Implement Details a) Fine-tuning Details.:We performed our fine-tuning experiments using the LLaMA Factory framework. We em- ployed a full fine-tuning ...

    arXiv 2025

  73. [73]

    How to Access:https://doi.org/10.5281/zenodo.17906756

    work page doi:10.5281/zenodo.17906756

  74. [74]

    •Recommended:NVIDIA H100 (80 GB), 128 GB RAM, 200 GB storage (used in paper)

    Hardware Dependencies: •Minimal:NVIDIA A100 (80 GB), 64 GB RAM, 140 GB storage. •Recommended:NVIDIA H100 (80 GB), 128 GB RAM, 200 GB storage (used in paper). Batch size is adjustable for different GPU memory

  75. [75]

    Complete list inrequirements.txt

    Software Dependencies: •Core:Python 3.10+, CUDA 12.1+ •Key packages:PyTorch 2.5.1, Transformers 4.57.0, Datasets 4.1.1, FAISS-CPU 1.9.0, Sentence-Transformers 5.1.1. Complete list inrequirements.txt. LLaMA-Factory is used for fine-tuning

  76. [76]

    •Reference dataset:Dolly-15k

    Benchmarks: •Primary dataset:HealthCareMagic-100k. •Reference dataset:Dolly-15k. •Model:LLaMA-3.2-3B-Instruct (requires HuggingFace account and license). •Optional:CNN-DM, iCliniq, MedInstruct-52k, TOFU. B. Installation and Configuration Detailed installation instructions inREADME.md. Summary:

  77. [77]

    Create conda environment with Python 3.10

  78. [78]

    Install dependencies:pip install -r requirements.txt

  79. [79]

    Install LLaMA-Factory (see README)

  80. [80]

    Experiment Workflow The workflow consists of four stages:

    Login to HuggingFace:huggingface-cli login C. Experiment Workflow The workflow consists of four stages:

Showing first 80 references.