Pith. sign in

REVIEW 4 major objections 6 minor 47 references

Membership Inference Attacks on Sequence Models

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that membership inference on sequence models is dramatically improved by modeling per-token loss correlations instead of averaging losses into a single score.

desk verdict A useful multivariate extension of LiRA for sequence models, but the 'consistently improves' claim outruns the evidence at low FPR. read the letter →

arxiv 2506.05126 v1 pith:GMIQHUIW submitted 2025-06-05 cs.CR cs.LG

Lorenzo Rossi , Michael Aerni , Jie Zhang , Florian Tramèr This is my paper
classification cs.CRcs.LG
keywords membershipinferenceattackssequencemodelsper-tokenlossescovarianceestimationOASshrinkagememorizationauditingautoregressiveprivacyleakage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Membership inference attacks on sequence models often fail because they compress a sequence into one average loss, treating tokens as independent. This paper adapts the Likelihood Ratio Attack to model the full vector of per-token losses as a multivariate Gaussian, estimating its covariance with OAS shrinkage. In language-model experiments, the adapted attack uncovers up to 87.5 times more memorized samples at a false-positive rate of 0.01%, and sharing one covariance between member and nonmember distributions helps when few shadow models are available. Because auditing large language models for memorized sensitive data is currently unreliable and shadow-model training is expensive, the practical point is that stronger audits are possible with nearly no extra cost.

What carries the argument

The central object is the multivariate Gaussian likelihood ratio over per-token loss vectors, which replaces LiRA's univariate likelihood ratio over the summed loss. Given a target sequence, the attack forms the vector of per-token log losses, estimates IN and OUT distributions over those vectors from shadow models trained with and without the sequence, and scores membership by the ratio of the two multivariate Gaussian densities. The covariance carrying the correlation information is estimated by OAS shrinkage, a data-adaptive convex combination of the sample covariance and a scaled identity that minimizes expected mean-squared error; the paper further doubles the sample size for covariance estimation by sharing one covariance matrix between the IN and OUT classes.

What would settle it

Train an autoregressive model on tokens that are statistically independent of each other, then run the OAS Shared attack and compare it with the Univariate Class-Wise baseline. If the correlation-aware attack does not beat the baseline, the gains really come from modeling correlations; if it still beats the baseline, correlation modeling is not the active mechanism and the paper's explanation is wrong.

Watch

Extended reading notes

Core claim

The paper's central claim is that within-sequence correlations carry membership signal that the average loss discards, and that exploiting them significantly boosts attack performance. Concretely, the authors argue that a naive application of LiRA to autoregressive models implicitly assumes independent per-token losses, which is false for language and images; they replace the univariate Gaussian over the sequence loss with a multivariate Gaussian over the per-token loss vector. To make this estimation practical with few shadow models, they use the Oracle Approximating Shrinkage estimator for the covariance, and they find empirically that a single shared covariance for members and nonmembers beats separate class-wise covariance estimates. Across LSTM and Pythia 1b language models, and a PixelCNN++ image generator, the sequence-aware variants match or beat the univariate baseline, with the largest gains at the low false-positive rates used in privacy auditing.

Load-bearing premise

The attack assumes that a model's per-token loss vector follows a multivariate Gaussian distribution, and that its covariance can be estimated reliably from a small number of shadow models using OAS shrinkage.

Editorial extensions

If this is right

  • Sequence-aware LiRA consistently uncovers more memorization than the average-loss baseline, often by an order of magnitude, at TPR@0.01% FPR.
  • Shared-covariance attacks attain comparable detection with fewer shadow models, reducing the computational cost of privacy audits.
  • The improvements transfer beyond text: for the PixelCNN++ autoregressive image generator, only the correlation-aware OAS variant beat random guessing.
  • Worst-case canaries constructed from random token sequences are detected at high true-positive rates, showing that audits can reliably flag atypical memorized strings.
  • Length-reduction strategies such as keeping only the two tokens with the smallest loss can further improve the attack in some settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If per-token covariance is what carries the signal, then membership audits could be sharpened further by selecting tokens whose loss is anomalous given their neighbors, a conditional analogue of the Min-K% selection rule.
  • For very long sequences, the full T-by-T covariance may be too large, so a structured low-rank or banded covariance prior is a natural testable extension; the paper's results suggest that a strong inductive bias can substitute for many shadow models.
  • These findings imply that privacy defenses aimed at reducing average loss are insufficient: an auditor can still detect membership from the correlation structure, so defenses would need to flatten the per-token loss covariance, not just the mean loss.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper argues that standard membership inference attacks on sequence models, specifically LiRA, fail because they treat per-token losses as independent and ignore within-sequence correlations. The authors adapt LiRA by modeling the vector of per-token losses as a multivariate Gaussian, estimating the covariance with an Oracle Approximating Shrinkage (OAS) estimator, and also explore shared versus class-wise covariance matrices and length-reduction strategies. Experiments on an LSTM, a fine-tuned Pythia 1b model, and a PixelCNN++ image generator show that sequence-aware variants often improve TPR at low FPR, sometimes by large factors. The paper concludes that correlation-aware modeling consistently improves memorization audits without additional computational costs.

Significance. The conceptual idea is appealing: replacing the scalar loss in LiRA with a multivariate Gaussian over per-token losses is a natural and potentially impactful practical recipe for privacy auditing of sequence models. The experimental design covers three model families, average- and worst-case canaries, and a sweep over the number of shadow models, which is valuable for practitioners. The use of OAS shrinkage to make covariance estimation feasible with few shadow models is well motivated, and the paper is transparent about the large design space it explores. However, the current evidence does not support the universal claims made in the abstract and introduction; the results are setting-dependent and lack uncertainty quantification. The paper's main value at this stage is the careful mapping of the design space rather than a validated single best attack.

major comments (4)
  1. [Abstract and Section 4.2] The claim that correlation-aware adaptations 'consistently improve' and 'never truly hurts' is not supported by the paper's own results. In Table 2, LSTM worst-case, OAS Class-Wise TPR@0.1% FPR is 23.19% versus 91.52% for Independent Class-Wise and 92.51% for Univariate Class-Wise; OAS Shared is 70.74% versus 95.82% for Univariate Shared. Figure 5 shows that for 8 and 16 shadow models, Univariate Shared outperforms all multivariate variants. The abstract and conclusion must be revised to state that the improvement is setting- and regime-dependent, or the authors must introduce and validate a specific variant that is consistently at least as good as the univariate baseline.
  2. [Sections 4.1 and Table 2] The headline TPR@0.01% FPR numbers are reported without uncertainty quantification, even though at 0.01% FPR with 10,000 canaries the expected number of false positives is only 1. For example, the difference between 0.04% and 0.87% in the LSTM average-case row (Independent Shared versus OAS Shared) corresponds to 4 versus 87 true positives and is likely subject to large binomial sampling noise; the '87.5x' improvement in Figure 1 rests on such point estimates. Please provide confidence intervals, multiple seeds, or exact counts with bootstrap intervals for the key comparisons, and avoid strong universal wording based on point estimates.
  3. [Table 2 caption] The caption states that 'The univariate attacks always under-perform the sequence-aware attacks', but this is directly contradicted by the LSTM worst-case column: Univariate Shared reaches 95.82% TPR@0.1% FPR versus 70.74% for OAS Shared, and 88.25% TPR@0.01% FPR versus 0.02% for Independent Class-Wise. Additionally, Figure 1 selects OAS Shared as 'Our Approach' even though this variant is not consistently the best across the table (e.g., in Pythia average-case, Independent Shared outperforms OAS Shared at both reported FPR levels). The paper should either define a fixed final attack and evaluate it fairly against all baselines or transparently present the full set of variants without a single cherry-picked representative.
  4. [Appendix G and Section 4.2] The length-reduction strategy 'Min' with length 2 is reported as giving the best MIA in the LSTM average-case setting, but this choice appears to be made after inspecting the same evaluation data used for the headline claims. This introduces a selection effect that is not accounted for in the reported TPR numbers. Please clarify whether the length-2 Min configuration was selected on a held-out set or justify its use as a fixed design choice rather than a post-hoc best result.
minor comments (6)
  1. [Figures 2 and 6] The legend label 'Indepedent' is misspelled in Figures 2 and 6; it should read 'Independent'.
  2. [Section 3.1] The words 'homoscedascity' and 'heteroscedascity' are misspelled; they should be 'homoscedasticity' and 'heteroscedasticity'.
  3. [Section 3.1] The notation N(S(f;x)|...) overloads the normal density; please clarify whether this denotes a probability density function and define S for the multivariate case explicitly, including how the per-token score vector is constructed.
  4. [Section 4.2] The sentence 'The optimal attack method and length reduction strategy vary depending on the specific setting' has a subject-verb agreement issue; it should be 'vary' instead of 'varies' only if the compound subject is treated as plural, and the sentence should be rephrased for clarity.
  5. [Appendix C] The statement that PixelCNN++ is 'slightly under-trained' is unquantified; reporting the final training loss or a sample of generated images would help the reader judge how challenging the setting is.
  6. [Overall] No code or data release is mentioned; making the implementation and experimental configuration publicly available would substantially improve reproducibility of the exact numbers in Tables 2 and Figures 4-5.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the attack's likelihood ratio is computed from shadow-model distributions and an external covariance estimator, not fitted to the target result.

full rationale

The derivation chain is self-contained in the relevant sense. LiRA's likelihood ratio is built from IN/OUT distributions estimated on shadow models with leave-one-out evaluation, so the attack score is not fitted to the target's membership label. The multivariate-Gaussian modeling of per-token losses is explicitly stated as an assumption rather than derived from the outcome, and the OAS covariance estimator (Eq. 1) is imported from external statistics literature (Chen et al.), not from the authors' own prior work. The central comparisons are empirical evaluations on held-out canaries with 64 or 484 shadow models, not quantities that reduce by construction to the method's inputs. The paper itself acknowledges the main caveat: 'The optimal attack method and length reduction strategy vary depending on the specific setting,' and Figure 5 shows the Univariate Shared attack is slightly better for fewer than 16 shadow models, contradicting the abstract's 'consistently improve' phrasing. That is a robustness or generalization concern, not circularity. The many self-citations (LiRA [5], impossibility results [4], audit-metric discussions [18]) are background or standard published baselines and none is load-bearing in the sense of being the sole justification for a forced conclusion. No step in the paper exhibits the required reduction of a prediction to its own inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a modeling assumption (multivariate Gaussian per-token losses) and on model-selection choices made on the same canaries used for evaluation. No new physical entities are introduced.

free parameters (2)
  • Min sequence length for length reduction = 2 (LSTM average case)
    In Appendix G, the Min strategy with reduced sequence length 2 is selected because it yields the highest TPR@0.01% FPR on the evaluation canaries; this is a hyperparameter chosen after seeing the results.
  • Covariance structure choice = OAS Shared, Independent Shared, etc.
    The attack variant and the decision to pool IN and OUT covariances are selected based on empirical performance on the same canaries (Figures 3 to 5, Table 2), so the reported best results reflect model selection on the test data.
assumptions (4)
  • domain assumption Per-token loss vectors for a sample across shadow models follow a multivariate Gaussian distribution.
    Underpins the likelihood ratio test in Section 3.1; non-Gaussian losses would make the test suboptimal.
  • domain assumption The covariance matrix of per-token losses is shared between IN and OUT distributions, or pooling samples improves estimation.
    Used by Shared variants (Section 3.2) to double the sample count for covariance estimation; empirically motivated by Figure 3.
  • domain assumption Shadow models are representative of the victim model's loss distribution.
    Standard LiRA assumption (Section 2.2) required for the estimated IN and OUT distributions to apply to the target model.
  • standard math The Neyman-Pearson lemma makes the likelihood ratio test uniformly most powerful under the Gaussian model.
    Invoked in Section 2.2 to justify LiRA's hypothesis test.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Membership Inference Attacks on Sequence Models." pith.science (2026). https://pith.science/paper/GMIQHUIW

@misc{pith2026250605126,
  author       = {Pith},
  title        = {Pith review of: Membership Inference Attacks on Sequence Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GMIQHUIW}},
  note         = {Machine review of arXiv:2506.05126}
}
read the original abstract

Sequence models, such as Large Language Models (LLMs) and autoregressive image generators, have a tendency to memorize and inadvertently leak sensitive information. While this tendency has critical legal implications, existing tools are insufficient to audit the resulting risks. We hypothesize that those tools' shortcomings are due to mismatched assumptions. Thus, we argue that effectively measuring privacy leakage in sequence models requires leveraging the correlations inherent in sequential generation. To illustrate this, we adapt a state-of-the-art membership inference attack to explicitly model within-sequence correlations, thereby demonstrating how a strong existing attack can be naturally extended to suit the structure of sequence models. Through a case study, we show that our adaptations consistently improve the effectiveness of memorization audits without introducing additional computational costs. Our work hence serves as an important stepping stone toward reliable memorization audits for large sequence models.

Figures

Figures reproduced from arXiv: 2506.05126 by the authors.

Figure 1
Figure 1. Strong membership inference for sequence models must consider correlations between sequence elements. We apply LiRA, a state-of-the-art MIA, to mea￾sure memorization in language modeling. First, we use LiRA as-is, where membership guesses only depend on a sample’s loss (“Naive Approach”). This approach uncovers only a small amount of memorization, as indicated by TPR (True Positive Rate) values close to the FPR (Fal… view at source ↗
Figure 2
Figure 2. Assuming independence between per-token losses can yield a large approximation error. We use the full covariance MLE (with 484 shadow models) as the gold standard, and measure the distance in Frobenius norm (“Covariance Approximation Error”) to other models’ estimates. As the number of shadow models increases, we analyze this error using 1,000 average-case canaries from the IN case of Pythia 1b with a sequence lengt… view at source ↗
Figure 5
Figure 5. The Univariate Shared attack is slightly better for small numbers of shadow models (< 16), but it is significantly worse for large numbers of shadow models. The reported TPR@0.01% FPR results illustrate the perfor￾mance of various attacks as the number of shadow models increases, based on experiments conducted with Pythia 1b and 10,000 average-case canaries. important to analyze the covariance matrix to understand w… view at source ↗
Figures from the paper (9 more)
Figure 1
Figure 1. Figure 1: The two groups of bars refer to the LSTM, and the Pythia 1b, which are the two language settings analyzed, using average-case canaries. As described in Sec￾tion 4.1, we used 10,000 samples and 64 shadow models. The ”Naive Approach”, which corresponds to LiRA as-is, whe…
Figure 6
Figure 6. Figure 6: The OUT case follows the same trends as the IN case. Setting Epochs Learning Rate Weight decay Batch Size Hidden Dimension Sequence Length LSTM 10 10−3 0.0 64 192 128 Pythia 1b 7 10−4 10−4 16 / 128 Pixel-CNN++ 50 10−3 0.0 192 192 1024 TABLE 1: The selected hyperparamet…
Figure 7
Figure 7. Figure 7: OAS is the only approach that performs better than random guessing. The TPR @ 1% FPR using Pixel￾CNN++ for average-case canaries. OAS) obtain higher TPRs across all the settings. For the image case, all the attacks have a close to random guessing performance due to the…
Figure 9
Figure 9. Figure 9: Using the Naive Approach (Univariate) gives suboptimal results. Comparing the true positive rate vs. false positive rate for different ways to model the LiRA IN and OUT distributions in the class-wise case. On the left, we use an LSTM, and on the right Pythia 1b using …
Figure 10
Figure 10. Figure 10: Using Naive Approach (Univariate) gives suboptimal results, also when considering a shared covariance matrix. Comparing the true positive rate vs. false positive rate for different ways to model the LiRA IN and OUT distributions in the shared case. On the left, we use…
Figure 11
Figure 11. Figure 11: Pythia - Average-case canaries. TPR @ 0.01% FPR for different length reduction strategies on Pythia 1b with the average-case canaries. The black lines represent the baselines, where no grouping strategy is applied, and all the tokens in the sequence are used. The dash…
Figure 12
Figure 12. Figure 12: Pythia - Worst-case canaries. TPR @ 0.01% FPR for different length reduction strategies on Pythia 1b with the worst-case canaries. The black lines represent the baselines, where no grouping strategy is applied, and all the tokens in the sequence are used. The dashed l…
Figure 13
Figure 13. Figure 13: LSTM - Average-case canaries. TPR @ 0.01% FPR for different length reduction strategies on LSTM with the average-case canaries. The black lines represent the baselines, where no grouping strategy is applied, and all the tokens in the sequence are used. The dashed line…
Figure 14
Figure 14. Figure 14: LSTM - Worst-case canaries. TPR @ 0.01% FPR for different length reduction strategies on LSTM with the worst-case canaries. The black lines represent the baselines, where no grouping strategy is applied, and all the tokens in the sequence are used. The dashed lines re…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 35 canonical work pages

  1. [1]

    Position: Considerations for differentially private learning with large-scale public pretraining,

    F. Tram `er, G. Kamath, and N. Carlini, “Position: Considerations for differentially private learning with large-scale public pretraining,” in Forty-first Interna- tional Conference on Machine Learning , 2022

    work page 2022

  2. [2]

    What does it mean for a language model to preserve privacy?

    H. Brown, K. Lee, F. Mireshghallah, R. Shokri, and F. Tram`er, “What does it mean for a language model to preserve privacy?” in Proceedings of the 2022 ACM conference on fairness, accountability, and trans- parency, 2022, pp. 2280–2292

    work page 2022

  3. [3]

    Membership inference attacks against machine learn- ing models,

    R. Shokri, M. Stronati, C. Song, and V . Shmatikov, “Membership inference attacks against machine learn- ing models,” in 2017 IEEE symposium on security and privacy (SP). IEEE, 2017, pp. 3–18

    work page 2017

  4. [4]

    Membership inference attacks cannot prove that a model was trained on your data,

    J. Zhang, D. Das, G. Kamath, and F. Tram `er, “Membership inference attacks cannot prove that a model was trained on your data,” ArXiv preprint , vol. abs/2409.19798, 2024. [Online]. Available: https: //arxiv.org/abs/2409.19798

    arXiv 2024

  5. [5]

    Membership inference attacks from first principles,

    N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer, “Membership inference attacks from first principles,” in 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022, pp. 1897–1914

    work page 2022

  6. [6]

    Palm 2 technical report,

    R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen et al. , “Palm 2 technical report,” ArXiv preprint , vol. abs/2305.10403, 2023. [Online]. Available: https://arxiv.org/abs/2305.10403

    arXiv 2023

  7. [7]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,

    M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, J.-b. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser et al., “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,” ArXiv preprint , vol. abs/2403.05530, 2024. [Online]. Available: https://arxiv.org/abs/2403.05530

    arXiv 2024

  8. [8]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al., “Gpt-4 technical report,” ArXiv preprint , vol. abs/2303.08774, 2023. [Online]. Available: https://arxiv.org/abs/2303.08774

    arXiv 2023

Show all 47 references
  1. [9]

    Do membership inference attacks work on large language models?

    M. Duan, A. Suri, N. Mireshghallah, S. Min, W. Shi, L. Zettlemoyer, Y . Tsvetkov, Y . Choi, D. Evans, and H. Hajishirzi, “Do membership inference attacks work on large language models?” in Conference on Lan- guage Modeling (COLM) , 2024

    2024

  2. [10]

    The privacy onion effect: Memorization is relative,

    N. Carlini, M. Jagielski, C. Zhang, N. Papernot, A. Terzis, and F. Tramer, “The privacy onion effect: Memorization is relative,” Advances in Neural Infor- mation Processing Systems, vol. 35, pp. 13 263–13 276, 2022

    2022

  3. [11]

    Quantifying memorization across neu- ral language models,

    N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tram `er, and C. Zhang, “Quantifying memorization across neu- ral language models,” in The Eleventh International Conference on Learning Representations . OpenRe- view, 2023

    2023

  4. [12]

    Extracting training data from large language models,

    N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-V oss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson et al., “Extracting training data from large language models,” in 30th USENIX Security Symposium (USENIX Security 21) , 2021, pp. 2633– 2650

    2021

  5. [13]

    Scalable extraction of training data from (production) language models,

    M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tram`er, and K. Lee, “Scalable extraction of training data from (production) language models,” ArXiv preprint , vol. abs/2311.17035, 2023. [Online]. Available: https:/...

    2023 arXiv

  6. [14]

    Extracting training data from document- based vqa models,

    F. Pinto, N. Rauschmayr, F. Tram `er, P. Torr, and F. Tombari, “Extracting training data from document- based vqa models,” in Proceedings of the 41st Interna- tional Conference on Machine Learning, ser. ICML’24. JMLR.org, 2024

    2024

  7. [15]

    Diffusion art or digital forgery? investigating data replication in diffusion models,

    G. Somepalli, V . Singla, M. Goldblum, J. Geiping, and T. Goldstein, “Diffusion art or digital forgery? investigating data replication in diffusion models,” in Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2023, pp. 6048– 6058

    2023

  8. [16]

    Extracting training data from diffusion models,

    N. Carlini, J. Hayes, M. Nasr, M. Jagielski, V . Sehwag, F. Tram `er, B. Balle, D. Ippolito, and E. Wallace, “Extracting training data from diffusion models,” in 32nd USENIX Security Symposium (USENIX Security 23). Anaheim, CA: USENIX Association, 2023, pp. 5253–5270. [Online]...

    2023

  9. [17]

    Measuring non-adversarial re- production of training data in large language models,

    M. Aerni, J. Rando, E. Debenedetti, N. Carlini, D. Ip- polito, and F. Tram `er, “Measuring non-adversarial re- production of training data in large language models,” arXiv preprint arXiv:2411.10242 , 2024

    2024 arXiv

  10. [18]

    Evaluations of machine learning privacy defenses are misleading,

    M. Aerni, J. Zhang, and F. Tram `er, “Evaluations of machine learning privacy defenses are misleading,” in ACM SIGSAC Conference on Computer and Commu- nications Security (CCS) , 2024, pp. 1271–1284

    2024

  11. [19]

    Does learning require memorization? a short tale about a long tail,

    V . Feldman, “Does learning require memorization? a short tale about a long tail,” in Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, 2020, pp. 954–959

    2020

  12. [20]

    Memorization in self- supervised learning improves downstream generaliza- tion,

    W. Wang, M. A. Kaleem, A. Dziedzic, M. Backes, N. Papernot, and F. Boenisch, “Memorization in self- supervised learning improves downstream generaliza- tion,” in The Twelfth International Conference on Learning Representations, 2024

    2024

  13. [21]

    Privacy risk in machine learning: Analyzing the con- nection to overfitting,

    S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the con- nection to overfitting,” in 2018 IEEE 31st computer security foundations symposium (CSF) . IEEE, 2018, pp. 268–282

    2018

  14. [22]

    Revisiting membership inference under re- alistic assumptions,

    B. Jayaraman, L. Wang, K. Knipmeyer, Q. Gu, and D. Evans, “Revisiting membership inference under re- alistic assumptions,” Proceedings on Privacy Enhanc- ing Technologies, 2021

    2021

  15. [23]

    Detecting credential spearphishing in enter- prise settings,

    G. Ho, A. Sharma, M. Javed, V . Paxson, and D. Wag- ner, “Detecting credential spearphishing in enter- prise settings,” in 26th USENIX security symposium (USENIX security 17) , 2017, pp. 469–485

    2017

  16. [24]

    The pitfalls of average-case differential privacy,

    T. Steinke and J. Ullman, “The pitfalls of average-case differential privacy,” DifferentialPrivacy.org, 2020, https://differentialprivacy.org/average-case-dp/

    2020

  17. [25]

    Privacy au- diting with one (1) training run,

    T. Steinke, M. Nasr, and M. Jagielski, “Privacy au- diting with one (1) training run,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  18. [26]

    Detecting pretraining data from large language models,

    W. Shi, A. Ajith, M. Xia, Y . Huang, D. Liu, T. Blevins, D. Chen, and L. Zettlemoyer, “Detecting pretraining data from large language models,” 2023

    2023

  19. [27]

    Min-k%++: Improved baseline for detecting pre-training data from large language models,

    J. Zhang, J. Sun, E. Yeats, Y . Ouyang, M. Kuo, J. Zhang, H. Yang, and H. Li, “Min-k%++: Improved baseline for detecting pre-training data from large language models,” arXiv preprint arXiv:2404.02936 , 2024

    2024 arXiv

  20. [28]

    Membership inference attacks against language models via neigh- bourhood comparison,

    J. Mattern, F. Mireshghallah, Z. Jin, B. Schoelkopf, M. Sachan, and T. Berg-Kirkpatrick, “Membership inference attacks against language models via neigh- bourhood comparison,” in Findings of the Association for Computational Linguistics: ACL 2023 , 2023, pp. 11 330–11 343

    2023

  21. [29]

    White-box vs black-box: Bayes optimal strategies for membership inference,

    A. Sablayrolles, M. Douze, C. Schmid, Y . Ollivier, and H. J ´egou, “White-box vs black-box: Bayes optimal strategies for membership inference,” in Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA , se...

    2019

  22. [30]

    On the importance of difficulty calibration in membership inference attacks,

    L. Watson, C. Guo, G. Cormode, and A. Sablayrolles, “On the importance of difficulty calibration in membership inference attacks,” in The Tenth International Conference on Learning Represen- tations, ICLR 2022, Virtual Event, April 25-29,

    2022

  23. [31]

    Truth serum: Poisoning machine learning models to reveal their secrets,

    F. Tram `er, R. Shokri, A. San Joaquin, H. Le, M. Jagiel- ski, S. Hong, and N. Carlini, “Truth serum: Poisoning machine learning models to reveal their secrets,” in Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security , 2022, pp. 2779–2792

    2022

  24. [32]

    Low- cost high-power membership inference attacks,

    S. Zarifzadeh, P. Liu, and R. Shokri, “Low- cost high-power membership inference attacks,” in Proceedings of the 41st International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. ...

    2024

  25. [33]

    Ix. on the problem of the most efficient tests of statistical hypotheses,

    J. Neyman and E. S. Pearson, “Ix. on the problem of the most efficient tests of statistical hypotheses,” Philo- sophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character , vol. 231, no. 694-706, pp. 289– 337, 1933

    1933

  26. [34]

    The neural autore- gressive distribution estimator,

    H. Larochelle and I. Murray, “The neural autore- gressive distribution estimator,” in Proceedings of the fourteenth international conference on artificial intelli- gence and statistics. JMLR Workshop and Conference Proceedings, 2011, pp. 29–37

    2011

  27. [35]

    Scalable pre-training of large autoregres- sive image models,

    A. El-Nouby, M. Klein, S. Zhai, M. ´A. Bautista, V . Shankar, A. T. Toshev, J. M. Susskind, and A. Joulin, “Scalable pre-training of large autoregres- sive image models,” inForty-first International Confer- ence on Machine Learning , 2024. [Online]. Available: https://openrevi...

    2024

  28. [36]

    PixelCNN++: Improving the pixelCNN with discretized logistic mixture likelihood and other modifications,

    T. Salimans, A. Karpathy, X. Chen, and D. P. Kingma, “PixelCNN++: Improving the pixelCNN with discretized logistic mixture likelihood and other modifications,” in International Conference on Learning Representations , 2017. [Online]. Available: https://openreview.net/forum?id=...

    2017

  29. [37]

    Some recent advances in forecasting and control,

    G. E. Box and G. M. Jenkins, “Some recent advances in forecasting and control,” Journal of the Royal Sta- tistical Society. Series C (Applied Statistics) , vol. 17, no. 2, pp. 91–109, 1968

    1968

  30. [38]

    Protein design and vari- ant prediction using autoregressive generative models,

    J.-E. Shin, A. J. Riesselman, A. W. Kollasch, C. McMahon, E. Simon, C. Sander, A. Manglik, A. C. Kruse, and D. S. Marks, “Protein design and vari- ant prediction using autoregressive generative models,” Nature communications, vol. 12, no. 1, p. 2403, 2021

    2021

  31. [39]

    Long short-term memory,

    S. Hochreiter, “Long short-term memory,” Neural Computation MIT-Press, 1997

    1997

  32. [40]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems , 2017

    2017

  33. [41]

    Pythia: A suite for analyzing large language models across training and scaling,

    S. Biderman, H. Schoelkopf, Q. G. Anthony, H. Bradley, K. O’Brien, E. Hallahan, M. A. Khan, S. Purohit, U. S. Prashanth, E. Raff et al. , “Pythia: A suite for analyzing large language models across training and scaling,” in International Conference on Machine Learning. PMLR, 2...

    2023

  34. [42]

    Shrinkage algorithms for mmse covariance estima- tion,

    Y . Chen, A. Wiesel, Y . C. Eldar, and A. O. Hero, “Shrinkage algorithms for mmse covariance estima- tion,” IEEE transactions on signal processing , vol. 58, no. 10, pp. 5016–5029, 2010

    2010

  35. [43]

    Enhanced membership inference at- tacks against machine learning models,

    J. Ye, A. Maddi, S. K. Murakonda, V . Bindschaedler, and R. Shokri, “Enhanced membership inference at- tacks against machine learning models,” in Proceed- ings of the 2022 ACM SIGSAC Conference on Com- puter and Communications Security , 2022, pp. 3093– 3106

    2022

  36. [44]

    Gemini: a family of highly capable multimodal models,

    G. Team, R. Anil, S. Borgeaud, Y . Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth et al., “Gemini: a family of highly capable multimodal models,” ArXiv preprint , vol. abs/2312.11805, 2023. [Online]. Available: https://arxiv.org/abs/2312.11805

    2023 arXiv

  37. [45]

    Personalizing dialogue agents: I have a dog, do you have pets too?

    S. Zhang, E. Dinan, J. Urbanek, A. Szlam, D. Kiela, and J. Weston, “Personalizing dialogue agents: I have a dog, do you have pets too?” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), I. Gurevych and Y . Miyao...

    2018

  38. [46]

    Decoupled weight decay regulariza- tion,

    I. Loshchilov, “Decoupled weight decay regulariza- tion,” arXiv preprint arXiv:1711.05101 , 2017. Appendix A Figure Setup Here, we describe the setup for the most relevant figures. Figure 1. The two groups of bars refer to the LSTM, and the Pythia 1b, which are the two languag...

    2017 arXiv

  39. [2022]

    [Online]

    OpenReview.net, 2022. [Online]. Available: https://openreview.net/forum?id=3eIrli0TwQ

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.