Pith. sign in

REVIEW 4 major objections 6 minor 62 references

Identifying Pre-training Data in LLMs: A Neuron Activation-Based Detection Framework

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A text's membership in an LLM's pre-training corpus can be detected by comparing which neurons it activates, with AUROC up to 99.7%.

desk verdict The neuron-activation detector is a real idea, but its headline accuracy rests on a benchmark that lets it spot machine rewriting, not pre-training membership. read the letter →

arxiv 2507.16414 v1 pith:RS4FH7HI submitted 2025-07-22 cs.AI

Hongyi Tang , Zhihao Zhu , Yi Yang This is my paper
classification cs.AI
keywords pre-trainingdatadetectionmembershipinferenceattackneuronactivationpatternslargelanguagemodelswhite-boxauditingCCNewsPDDbenchmarkcontaminationcopyrightverification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a text's membership in an LLM's pre-training corpus can be determined from which neurons the text activates, not just from loss or token probabilities. It introduces NA-PDD, which labels neurons as 'member' or 'non-member' by comparing activation frequencies on small reference sets of training and non-training text, then scores a test text by how much its activated neurons overlap with member neurons in the most discriminative layers. On the new CCNewsPDD benchmark, built from CCNews articles transformed by back-translation, masking, and rewriting to remove time drift, NA-PDD reaches AUROC up to 99.7% and outperforms nine prior methods. If correct, this gives open-source model owners and data owners a practical audit tool for copyright and contamination questions.

What carries the argument

The load-bearing object is the differential neuron activation profile: which feed-forward neurons in which layers fire above a threshold when a text passes through the model. The paper labels 'member neurons' as those whose activation frequency on a small training reference corpus is more than $\alpha$ times their frequency on a non-training reference corpus, and 'non-member neurons' by the symmetric condition. A test text is scored by the ratio of its overlap with member neurons to its overlap with non-member neurons, restricted to the $K$ layers where the member/non-member imbalance is largest. This machinery converts the abstract idea of memorization into a per-neuron, per-layer binary fingerprint that can be thresholded for membership prediction.

What would settle it

Pass both training and non-training CCNews articles through the same back-translation, masking, or rewriting pipeline and run NA-PDD on that matched pair; if the score separates them anyway, the reported signal is transformation artifacts, not membership.

Watch

Extended reading notes

Core claim

At the paper's core is the claim that training and non-training text activate different populations of neurons inside a transformer, and that this difference is a much stronger membership signal than surface statistics. Concretely, NA-PDD records post-activation outputs of feed-forward neurons above a threshold $\tau$, computes how often each neuron fires on a reference corpus of training data versus non-training data, and declares a neuron 'member' if its training frequency exceeds $\alpha$ times its non-training frequency (and 'non-member' for the reverse imbalance). For a query text $x$, the method computes the coincidence rate of $x$'s activated neurons with member neurons and with non-member neurons, averages these over the $K$ layers with the largest member/non-member imbalance, and predicts 'member' when the ratio $R(x,M)$ exceeds a threshold. The reported result is that this neuron-based score reaches 75.8% AUROC on WikiMIA, 57.2% on ArxivMIA, and 92.1–99.7% on the three CCNewsPDD variants, beating all nine compared baselines on every dataset.

Load-bearing premise

The evaluation assumes that the transformed CCNews articles are non-training solely because they were not in the corpus, not because the transformations make them look systematically different from the originals.

Editorial extensions

If this is right

  • Data owners can audit an open-source release: feed the suspect text and a small reference corpus through the model, and read off whether the text behaves like a member.
  • The same procedure transfers across model families such as Pythia, OPT, TinyLLaMA, and OpenLLaMA with no per-model feature engineering, only $\tau$, $\alpha$, and $K$ to set.
  • Because NA-PDD needs only about 200 reference samples and keeps its edge as models get larger, it is a practical white-box route to copyright verification rather than a research toy.
  • CCNewsPDD gives future pre-training-data-detection comparisons a shared, time-aligned test bed, so gains can be attributed to detection skill rather than release-date leakage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A matched-control experiment would strengthen the paper: if both member and non-member texts are put through the same back-translation, masking, or rewriting pipeline, the reported 92–99% AUROC may shrink, revealing how much of the signal is transformation artifacts rather than true membership.
  • If member neurons are the actual storage sites of memorized content, then zeroing or dampening those neurons should erase membership signals; that suggests a direct test and a possible machine-unlearning mechanism.
  • The neuron-activation fingerprint is unlikely to survive API-only access, so extending this style of detection to closed models would require approximating the activation pattern from outputs or from a distilled surrogate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes NA-PDD, a white-box method for pre-training data detection (PDD) in LLMs. NA-PDD records binary activation states of FFN neurons for a set of reference training and non-training samples, labels neurons as 'member' or 'non-member' by comparing activation frequencies, selects the most discriminative layers, and scores a test text by the ratio of its overlap with member versus non-member neurons. The authors also introduce CCNewsPDD, a benchmark whose non-training examples are generated by back-translation, BERT masking, and BART rewriting of CCNews articles from the same period as the training examples. Experiments across three benchmarks and several LLMs report AUC improvements over nine baselines, most notably 99.7% AUC on CCNewsPDD(prompt) for OPT-6.7B.

Significance. If the central claim holds, NA-PDD would be a practical white-box tool for auditing open-source LLMs and for copyright verification, and the paper provides a clear algorithmic framework with a plausible mechanistic motivation. The method is evaluated against nine baselines on three benchmarks, and the sensitivity analyses for hyperparameters are a useful contribution. However, the claim is not yet supported because the main benchmark's non-training class is constructed by machine transformations of training articles, which introduces a confound that the method can exploit; the paper provides no control that separates 'training membership' from 'original versus transformed text'. The absence of released code and data further limits verification. These issues are addressable with additional experiments, so the manuscript has a defensible core but requires major revision.

major comments (4)
  1. [Section 4, Table 2] The CCNewsPDD benchmark defines non-training data as back-translated, BERT-masked, or BART-rewritten versions of the same CCNews articles that serve as training data. This means the non-member class differs from the member class not by membership status but by systematic surface-form transformations (translationese, substitution artifacts, and generic rephrasing). The high AUC values, especially 99.7% on CCNewsPDD(prompt) for OPT, may therefore reflect discrimination between original and transformed text rather than pre-training membership. This concern is reinforced by the already-high baseline scores on the same benchmark (e.g., Probe Attack at 90.1% and Zlib at 80.7%), which indicate that the transformation signal is detectable by much simpler methods. A control using naturally occurring non-training articles from the same source and time period, without transformations, is needed to support the central claim.
  2. [Section 3.4, Eqs. (4)-(6)] The reference non-training set D_non used to define non-member neurons is itself the set of transformed CCNews articles. Consequently, the member/non-member neuron distinction is, by construction, a contrast between original CCNews text and machine-transformed CCNews text. If the model's neurons respond to translationese or rewrite artifacts, the score R(x, M) in Eq. (12) measures 'originalness' rather than membership. The paper should show that the selected member and non-member neurons are stable when D_non is replaced with genuinely non-training originals (e.g., articles from the same period that are not in the training set, or a held-out publisher split), and that the high AUC persists in that setting.
  3. [Section 6.1, Table 2] The large gap between performance on CCNewsPDD (92.4-99.7% AUC) and on the nonsynthetic benchmarks WikiMIA (71.6-75.8%) and ArxivMIA (57.2-59.3%) is consistent with the hypothesis that NA-PDD is exploiting transformation artifacts. The paper should explicitly analyze this discrepancy, for example by reporting which layers and neurons are selected on CCNewsPDD and whether they correlate with surface-level features like token frequency or translationese markers, and by evaluating on a non-transformed control benchmark with matched difficulty.
  4. [Section 5 and Appendix B] The implementation details are internally inconsistent: Section 5 reports activation threshold τ = 1.0, dominance threshold α = 1.5, and K = 10 discriminative layers, while Appendix B.2 reports τ = 1.5, α = 1.8, and K = 5, and Algorithm 1 lists 'α > 1.5' as an input. These discrepancies need to be resolved, and the final configuration used to produce Table 2 must be stated unambiguously. This is a reproducibility issue, though the sensitivity analysis suggests the results are not highly sensitive to these choices.
minor comments (6)
  1. [Table 1] The table caption and first column label the new benchmark 'CCNewsMIA (Ours)', while the rest of the paper calls it 'CCNewsPDD'. Please use a consistent name throughout.
  2. [Section 6.2, Figure 3] The text says 'reference data sizes (200-500 samples)' while Section 5 states that neuron identity uses 100 training and 100 non-training samples. Clarify whether the 200 samples in Figure 3 are total samples or per class, and reconcile with the 100+100 description.
  3. [Section 6.3, Figures 5 and 6] The sensitivity analysis for α reports AUC values varying by only 0.35% (90.90-91.25%), but the main results in Table 2 for the same dataset and model (Pythia-2.8B, CCNewsPDD(trans)) report 92.4%. Explain the source of this discrepancy, for example whether the sensitivity analysis is on a different split or test set.
  4. [Limitations section] The limitations paragraph appropriately acknowledges the white-box assumption and the need for a reference corpus, but it does not mention the transformation confound of CCNewsPDD. Please add a discussion of this limitation and any steps taken to mitigate it.
  5. [Section 3.6, Eq. (13)] The decision threshold θ is not defined in the main text; the paper reports AUC, which is threshold-independent, but for a deployable algorithm the choice of θ and how it would be set in practice should be discussed.
  6. [General] The paper does not state that code or data will be released, which is important for a benchmark and detection method of this type. Please include an availability statement.

Circularity Check

1 steps flagged · score 6.0 of 10

CCNewsPDD's 'non-training' class is machine-transformed training text, so NA-PDD's near-ceiling CCNewsPDD scores may reduce to original-vs-transformed discrimination rather than pre-training membership.

  1. self definitional [Section 4 Data Construction (CCNewsPDD definitions), with Section 3.4 Eq. (4) and Section 5 Implementation Details (reference corpora).]
    "To generate novel text that maintains authentic linguistic properties while ensuring that the target LLMs have not encountered it during training, we engage in a data transformation process. ... CCNewsPDD(prompt): This dataset generates non-training data through explicit instruction prompting. By directing the BART model (Lewis et al., 2020) to reformulate the original texts, we achieve comprehensive discourse-level rephrasings while preserving the core semantic content in the generated non-training data. ..."

    The non-training class in CCNewsPDD is not an independent corpus; it is produced by transforming the training-class CCNews articles via back-translation, BERT masking, or BART rewriting. The same transformation pipeline supplies the reference D_non used in Eq. (4) to label 'non-member' neurons, and it also defines the test-set 'non-training' examples. NA-PDD's decision statistic R(x,M) (Eq. 12) therefore separates original text from machine-transformed text, a distinction guaranteed by benchmark construction rather than by pre-training membership. The reported 99.7% AUC on CCNewsPDD(prompt) is thus compatible with detecting translationese, masking artifacts, or rephrasing style, and the paper provides no control with untransformed non-members to rule out this shortcut.

full rationale

NA-PDD itself is a supervised classifier trained on reference corpora, and its hyperparameters are tuned on held-out validation data, so the algorithm is not derivationally circular in the sense of fitting a parameter and calling it a prediction. The substantive circularity is in the CCNewsPDD benchmark: the negative class is defined as transformed versions of the positive class, and the same transformed distribution is used as the reference D_non when labeling member and non-member neurons in Eq. (4). Consequently the near-perfect AUCs on the three CCNewsPDD variants are partly an artifact of the benchmark definition, and the claim that neuron activation detects pre-training membership is confounded with detecting transformation artifacts. Results on WikiMIA and ArxivMIA are independent and much weaker (75.8% and 57.2% AUC), so the paper retains some independent content, but the headline CCNewsPDD evidence does not by itself establish the stated capability. No load-bearing self-citations or imported uniqueness theorems are present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method depends on several tuned hyperparameters and on assumptions about sparse activation, reference corpus adequacy, and the validity of transformed texts as non-training data. The last assumption is the most fragile and directly affects the benchmark's meaning.

free parameters (4)
  • activation threshold tau = 1.0 (main text) / 1.5 (Appendix B.2)
    Neurons with activation above tau are considered active; chosen by ablation over [0,2].
  • dominance threshold alpha = 1.5 (main text) / 1.8 (Appendix B.2)
    Controls stringency for labeling member/non-member neurons; chosen by ablation over [1.2,2.0].
  • number of discriminative layers K = 10 (main text) / 5 (Appendix B.2)
    Top-K layers by discriminative score are used for similarity; chosen by ablation over [1,32].
  • reference corpus size = 100 training + 100 non-training samples
    Used to label neurons; robustness tested with 200-500 samples but main results use 200 total.
assumptions (4)
  • domain assumption LLMs are sparsely activated, so a thresholded activation state is meaningful
    Invoked in Section 3.3 citing Wang et al. 2024 and Liu et al. 2024a; not verified for the tested models.
  • domain assumption Reference training and non-training corpora faithfully represent the true training/non-training distributions
    Section 3.4 uses 100+100 samples to label neurons; if reference is unrepresentative, neuron labels and predictions fail.
  • ad hoc to paper Transformed CCNews texts were never in the target models' pre-training corpora
    Section 4 asserts this to justify CCNewsPDD; back-translation, masking, and prompting may produce strings the model has seen through duplication or data processing, and the assumption is not empirically verified.
  • ad hoc to paper The transformation artifacts do not create detectable distributional differences
    Section 4 claims transformations preserve distribution, but the high AUC may come from translationese or rewrite style rather than membership; this is the key confound.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Identifying Pre-training Data in LLMs: A Neuron Activation-Based Detection Framework." pith.science (2026). https://pith.science/paper/RS4FH7HI

@misc{pith2026250716414,
  author       = {Pith},
  title        = {Pith review of: Identifying Pre-training Data in LLMs: A Neuron Activation-Based Detection Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RS4FH7HI}},
  note         = {Machine review of arXiv:2507.16414}
}
read the original abstract

The performance of large language models (LLMs) is closely tied to their training data, which can include copyrighted material or private information, raising legal and ethical concerns. Additionally, LLMs face criticism for dataset contamination and internalizing biases. To address these issues, the Pre-Training Data Detection (PDD) task was proposed to identify if specific data was included in an LLM's pre-training corpus. However, existing PDD methods often rely on superficial features like prediction confidence and loss, resulting in mediocre performance. To improve this, we introduce NA-PDD, a novel algorithm analyzing differential neuron activation patterns between training and non-training data in LLMs. This is based on the observation that these data types activate different neurons during LLM inference. We also introduce CCNewsPDD, a temporally unbiased benchmark employing rigorous data transformations to ensure consistent time distributions between training and non-training data. Our experiments demonstrate that NA-PDD significantly outperforms existing methods across three benchmarks and multiple LLMs.

Figures

Figures reproduced from arXiv: 2507.16414 by the authors.

Figure 1
Figure 1. An overview of NA-PDD. classification problem: A(x,M) → {0, 1} (1) where a prediction of 1 indicates that the model M has utilized the text x, while a prediction of 0 signifies that it has not. White-box settings. Following previous work, we assume access to the weights and activations of the target model. This framework is applicable in two real-world scenarios: 1) Model owners need to audit their models to prevent… view at source ↗
Figure 2
Figure 2. Comparison of AUC Values with Different Model Sizes (best viewed in color) [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Comparison of AUC Values with Different Training Data Sizes (best viewed in color). more distinctive neuron activation patterns for de￾tecting pretraining data. In contrast, the baseline method exhibits unstable behavior: after peaking at 78% AUC with the 410M model, its perfor￾mance sharply declines and stabilizes around 65% for models larger than 1.4B. This divergence sug￾gests that while traditional probe-based a… view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Comparison of AUC Values with Different [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparison of AUC Values with Different Numbers of Selected Discriminative Layers. strates the robustness of our method to the selection of α, with α = 1.5 recommended as a reliable de￾fault. Numbers of selected discriminative layers K. Our evaluation on the Pythia-2.8…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 25 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308--318

    work page 2016

  4. [4]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  5. [5]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

    arXiv 2023

  6. [6]

    Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, and 1 others. 2023. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pages 2397--2430. PMLR

    2023

  7. [7]

    Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In 2021 IEEE symposium on security and privacy (SP), pages 141--159. IEEE

    work page 2021

  8. [8]

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, and 1 others. 2021. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pages 2633--2650

    2021

Show all 62 references
  1. [9]

    Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. 2023. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pages 5253--5270

    2023

  2. [10]

    Kent K Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman. 2023. Speak, memory: An archaeology of books known to chatgpt/gpt-4. arXiv preprint arXiv:2305.00118

    2023 arXiv

  3. [11]

    Dingfan Chen, Ning Yu, Yang Zhang, and Mario Fritz. 2020. Gan-leaks: A taxonomy of membership inference attacks against generative models. In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security, pages 343--362

    2020

  4. [12]

    Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. 2021. Label-only membership inference attacks. In International conference on machine learning, pages 1964--1974. PMLR

    2021

  5. [13]

    Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023. Toxicity in chatgpt: Analyzing persona-assigned language models. arXiv preprint arXiv:2304.05335

    2023 arXiv

  6. [14]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

    2019

  7. [15]

    Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. 2024 a . Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841

    2024 arXiv

  8. [16]

    Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. 2024 b . Do membership inference attacks work on large language models? In First Conference on Language Modeling

    2024

  9. [17]

    Andr \'e V Duarte, Xuandong Zhao, Arlindo L Oliveira, and Lei Li. 2024. De-cop: Detecting copyrighted content in language models training data. arXiv preprint arXiv:2402.09910

    2024 arXiv

  10. [18]

    Emilio Ferrara. 2023. Should chatgpt be biased? challenges and risks of bias in large language models. arXiv preprint arXiv:2304.03738

    2023 arXiv

  11. [19]

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, and 1 others. 2020. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027

    2020 arXiv

  12. [20]

    Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462

    2020 arXiv

  13. [21]

    Xinyang Geng and Hao Liu. 2023. Openllama: An open reproduction of llama. URL: https://github. com/openlm-research/open\_llama

    2023

  14. [22]

    Felix Hamborg, Norman Meuschke, Corinna Breitinger, and Bela Gipp. 2017. https://doi.org/10.5281/zenodo.4120316 news-please: A generic news crawler and extractor . In Proceedings of the 15th International Symposium of Information Science, pages 218--223

    2017 doi

  15. [23]

    Nils Homer, Szabolcs Szelinger, Margot Redman, David Duggan, Waibhav Tembe, Jill Muehling, John V Pearson, Dietrich A Stephan, Stanley F Nelson, and David W Craig. 2008. Resolving individuals contributing trace amounts of dna to highly complex mixtures using high-density snp g...

    2008

  16. [24]

    Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. 2022. Membership inference attacks on machine learning: A survey. ACM Computing Surveys (CSUR), 54(11s):1--37

    2022

  17. [25]

    Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, and Neil Zhenqiang Gong. 2019. Memguard: Defending against black-box membership inference attacks via adversarial examples. In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pages 259--274

    2019

  18. [26]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361

    2020 arXiv

  19. [27]

    Hadas Kotek, Rikker Dockum, and David Sun. 2023. Gender bias and stereotypes in large language models. In Proceedings of the ACM collective intelligence conference, pages 12--24

    2023

  20. [28]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. https://doi.org/10.18653/v1/2020.acl-main.703 BART : Denoising sequence-to-sequence pre-training for natural language generation, translatio...

    2020 doi

  21. [29]

    Bo Liu, Ming Ding, Sina Shaham, Wenny Rahayu, Farhad Farokhi, and Zihuai Lin. 2021. When machine learning meets privacy: A survey and outlook. ACM Computing Surveys (CSUR), 54(2):1--36

    2021

  22. [30]

    James Liu, Pragaash Ponnusamy, Tianle Cai, Han Guo, Yoon Kim, and Ben Athiwaratkun. 2024 a . Training-free activation sparsity in large language models. arXiv preprint arXiv:2408.14690

    2024 arXiv

  23. [31]

    Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding, and Lianwen Jin. 2024 b . Datasets for large language models: A comprehensive survey. arXiv preprint arXiv:2402.18041

    2024 arXiv

  24. [32]

    Zhenhua Liu, Tong Zhu, Chuanyuan Tan, Bing Liu, Haonan Lu, and Wenliang Chen. 2024 c . Probing language models for pre-training data detection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1576--1587

    2024

  25. [33]

    Inbal Magar and Roy Schwartz. 2022. Data contamination: From memorization to exploitation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 157--165

    2022

  26. [34]

    Pratyush Maini and Anshuman Suri. 2025. Reassessing emnlp 2024’s best paper: Does divergence-based calibration for mias hold up? In The Fourth Blogpost Track at ICLR 2025

    2025

  27. [35]

    Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Sch \"o lkopf, Mrinmaya Sachan, and Taylor Berg-Kirkpatrick. 2023. Membership inference attacks against language models via neighbourhood comparison. arXiv preprint arXiv:2305.18462

    2023 arXiv

  28. [36]

    Matthieu Meeus, Shubham Jain, Marek Rei, and Yves-Alexandre de Montjoye. 2024. Did the neurons read your book? document-level membership inference for large language models. In 33rd USENIX Security Symposium (USENIX Security 24), pages 2369--2385

    2024

  29. [37]

    Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri. 2022. Quantifying privacy risks of masked language models using membership inference attacks. arXiv preprint arXiv:2203.03929

    2022 arXiv

  30. [38]

    Jakob M \"o kander, Jonas Schuett, Hannah Rose Kirk, and Luciano Floridi. 2024. Auditing large language models: a three-layered approach. AI and Ethics, 4(4):1085--1115

    2024

  31. [39]

    Maximilian Mozes, Xuanli He, Bennett Kleinberg, and Lewis D Griffin. 2023. Use of llms for illicit purposes: Threats, prevention measures, and vulnerabilities. arXiv preprint arXiv:2308.12833

    2023 arXiv

  32. [40]

    Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tram \`e r, and Katherine Lee. 2023. Scalable extraction of training data from (production) language models. arXiv preprint ar...

    2023 arXiv

  33. [41]

    Yonatan Oren, Nicole Meister, Niladri S Chatterji, Faisal Ladhak, and Tatsunori Hashimoto. 2023. Proving test set contamination in black-box language models. In The Twelfth International Conference on Learning Representations

    2023

  34. [42]

    Xudong Pan, Mi Zhang, Shouling Ji, and Min Yang. 2020. Privacy risks of general-purpose language models. In 2020 IEEE Symposium on Security and Privacy (SP), pages 1314--1331. IEEE

    2020

  35. [43]

    Apostolos Pyrgelis, Carmela Troncoso, and Emiliano De Cristofaro. 2017. Knock knock, who's there? membership inference on aggregate location data. arXiv preprint arXiv:1708.06145

    2017 arXiv

  36. [44]

    Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, and Herv \'e J \'e gou. 2019. White-box vs black-box: Bayes optimal strategies for membership inference. In International Conference on Machine Learning, pages 5558--5567. PMLR

    2019

  37. [45]

    Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. 2018. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv:1806.01246

    2018 arXiv

  38. [46]

    Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. 2023 a . Detecting pretraining data from large language models. arXiv preprint arXiv:2310.16789

    2023 arXiv

  39. [47]

    Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. 2023 b . Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations

    2023

  40. [48]

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3--18. IEEE

    2017

  41. [49]

    o rg Tiedemann, Mikko Aulamo, Daria Bakshandaeva, Michele Boggia, Stig-Arne Gr \

    J \"o rg Tiedemann, Mikko Aulamo, Daria Bakshandaeva, Michele Boggia, Stig-Arne Gr \"o nroos, Tommi Nieminen, Alessandro Raganato, Yves Scherrer, Ra \'u l V \'a zquez, and Sami Virpioja. 2024. Democratizing neural machine translation with opus-mt. Language Resources and Evalua...

    2024

  42. [50]

    Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. 2022. Memorization without overfitting: Analyzing the training dynamics of large language models. Advances in Neural Information Processing Systems, 35:38274--38290

    2022

  43. [51]

    Hongyu Wang, Shuming Ma, Ruiping Wang, and Furu Wei. 2024. Q-sparse: All large language models can be fully sparsely-activated. arXiv preprint arXiv:2407.10969

    2024 arXiv

  44. [52]

    Lauren Watson, Chuan Guo, Graham Cormode, and Alex Sablayrolles. 2021. On the importance of difficulty calibration in membership inference attacks. arXiv preprint arXiv:2111.08440

    2021 arXiv

  45. [53]

    Cheng Xu, Shuhao Guan, Derek Greene, M Kechadi, and 1 others. 2024. Benchmark data contamination of large language models: A survey. arXiv preprint arXiv:2406.04244

    2024 arXiv

  46. [54]

    Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. 2024 a . Machine unlearning of pre-trained large language models. In 62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024, pages 8403--8419. Association for Computat...

    2024

  47. [55]

    Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024 b . A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211

    2024

  48. [56]

    Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. 2018. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pages 268--282. IEEE

    2018

  49. [57]

    Jie Zhang, Debeshee Das, Gautam Kamath, and Florian Tram \`e r. 2024 a . Membership inference attacks cannot prove that a model was trained on your data. arXiv preprint arXiv:2409.19798

    2024 arXiv

  50. [58]

    Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Frank Yang, and Hai Li. 2024 b . Min-k\ arXiv preprint arXiv:2404.02936

    2024 arXiv

  51. [59]

    Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. 2024 c . Tinyllama: An open-source small language model. arXiv preprint arXiv:2401.02385

    2024 arXiv

  52. [60]

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, and 1 others. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068

    2022 arXiv

  53. [61]

    Weichao Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, and Xueqi Cheng. 2024 d . Pretraining data detection for large language models: A divergence-based calibration method. arXiv preprint arXiv:2409.14781

    2024 arXiv

  54. [62]

    Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1--38

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.