Pith. sign in

REVIEW 3 major objections 6 minor 47 references

Unlocking Post-hoc Dataset Inference with Synthetic Data

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Dataset inference can detect unauthorized training data using a synthetic held-out set generated from the suspect texts themselves, provided the natural-versus-synthetic shift is separated from membership by a dual-classifier t-test.

desk verdict A genuinely useful practical contribution to Dataset Inference that deserves a serious referee, but the paper overstates its false-positive control and needs a statistical cleanup before its litigation claims are credible. read the letter →

arxiv 2506.15271 v1 pith:VS6HEFFC submitted 2025-06-18 cs.LG cs.AI

Bihe Zhao , Pratyush Maini , Franziska Boenisch , Adam Dziedzic This is my paper · ORCID
classification cs.LGcs.AI
keywords datasetinferencemembershipattackssyntheticheld-outdatasuffixcompletionpost-hoccalibrationlargelanguagemodelscopyrightprotectionfalsepositiverate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dataset inference (DI) tries to determine whether a specific dataset was used to train a model, but established methods require a held-out set from the same distribution that is known to be absent from training—data that rarely exists in practice. The paper claims this bottleneck can be removed by generating the held-out set from the suspect texts themselves: a generator is fine-tuned on shuffled, segmented versions of the suspect data and completes shared prefixes, producing synthetic suffixes that are nearly indistinguishable from the real ones. It then adds a post-hoc calibration step in which a text-only classifier and a combined classifier (text plus the target model's membership-inference scores) are compared; any advantage of the combined classifier is counted as membership signal rather than generation artifact. On single-author blog posts and sixteen Pile subsets, member sets are detected with $p<0.05$ and non-member sets stay above threshold, so the paper concludes that DI becomes usable for copyright owners without a private in-distribution corpus.

What carries the argument

The load-bearing object is the suffix-completion generator used together with the dual-classifier difference t-test. The generator is a LoRA-finetuned LLM trained on the suspect texts after they are segmented into short sequences and randomly split, and it writes a synthetic continuation for each prefix; because original and synthetic suffixes share a prefix, they can be compared pair by pair. Two classifiers are then trained on the same suspect and held-out splits: $c_{\mathrm{text}}(x)$ sees only the text, while $c_{\mathrm{comb}}(x,\mathrm{MIA}(f(x)))$ also sees membership-inference scores from the target model. The test statistic compares $c_{\mathrm{comb}}(x_{\mathrm{val}})-c_{\mathrm{comb}}(x_{\mathrm{sus}})$ with $c_{\mathrm{text}}(x_{\mathrm{val}})-c_{\mathrm{text}}(x_{\mathrm{sus}})$, and only the combined classifier's extra edge is treated as membership evidence. A Sigmoid projection keeps the MIA weights in $(0,1)$ so that generated-text perplexity gaps cannot be summed into a false positive.

What would settle it

Run the complete pipeline on a non-member suspect set used to train the generator and both classifiers, repeating the paired t-test over many random splits; any rejection of the null at $p<0.05$ would show the method is detecting generation artifacts rather than membership. The paper's own Setting 6 shows that the uncalibrated t-test already produces false positives on non-member sets, so the decisive experiment is the same non-member corpus with the full calibration held fixed.

Watch

Extended reading notes

Core claim

The central claim, stated as the authors would state it, is that a synthetic held-out set generated from the suspect data itself can replace the missing in-distribution held-out set in dataset inference, provided the generation shift is measured and removed. The key construction is pairwise: each original suffix in the suspect set is paired with a synthetic suffix produced by a LoRA-finetuned generator from the same prefix, so the two texts differ only in continuation, not position. A text classifier trained on real-versus-generated suffixes captures the generation gap; a combined classifier that also receives the target model's MIA scores should beat the text classifier only when the suspect set was in training. The paper's difference comparison t-test formalizes this comparison, and the positive weight constraint stops perplexity-based MIA scores from pooling generation artifacts into a false signal. The paper reports that this procedure detects membership on every evaluated member set and refrains from accusing on every evaluated non-member set.

Load-bearing premise

The load-bearing premise is that the membership signal of the suspect set remains visible in the target model's membership-inference scores after the generator has been trained on the same texts, and that the combined classifier's advantage over the text-only classifier comes from that signal rather than from a distribution-based shortcut.

Editorial extensions

If this is right

  • A data owner can run dataset inference with only the suspect documents and black-box access to the target model, because the held-out set is generated on demand instead of being supplied privately.
  • Member detection reaches $p<0.05$ on every Pile subset tested, while non-member p-values stay above $0.1$, giving a concrete operating point for copyright discovery.
  • Statistical confidence grows with sample size: fewer than 1,000 paired samples reach significance on most datasets, and 2,000 samples give $p<0.01$ consistently.
  • The positive weight constraint alone changes false-positive outcomes on non-member sets, so the calibration logic can be reused as a guard against generation-induced MIA gaps.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This suggests the same dual-classifier logic could transfer to images, code, or audio, whenever a generator can be fine-tuned on the suspect set; the paper only tests text.
  • A natural stress test the paper does not run is adversarial selection of the suspect set to mimic the target model's public training distribution, which would probe whether the calibration survives a deliberate false-accusation attempt.
  • The paired suffix design also points to a cheap contamination-screening use: a few thousand target-model queries could flag whether a benchmark appeared in training, without any user-supplied held-out data.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper addresses the held-out data bottleneck in Dataset Inference (DI) by proposing to synthesize a held-out set from the suspect dataset itself. The generator is trained on a suffix-completion task over segmented snippets, and a dual-classifier difference t-test compares a text-only classifier with a membership-aware classifier, attributing any performance difference to membership rather than distribution shift. Experiments on a single-author blog corpus and on 15 Pile subsets report low p-values for member sets and mostly non-significant p-values for non-member sets, with ablations showing that suffix completion, post-hoc calibration, and the weight constraint each contribute to performance. If the assumptions hold, this removes the requirement for a real in-distribution held-out set, which is the main obstacle to applying DI in copyright discovery.

Significance. If the method's central assumption is valid, this is a practically valuable contribution: it enables dataset inference without access to real in-distribution held-out data, directly addressing a key limitation of prior DI methods. The experimental breadth (single-author to multi-author, code, legal, medical, multilingual data) is substantial, the ablations are informative, and the authors ship code. However, the statistical validation of the low-false-positive claim is not yet convincing: several non-member p-values in Table 4 are close to the 0.05 threshold, and the justification that any advantage of the combined classifier over the text-only classifier must come from membership rather than distribution shift is not established. The central idea is promising, but the current evidence requires strengthening before the reliability claims can be accepted.

major comments (3)
  1. [Section 4.2, Eq. (2), and Table 4] The key assumption that any performance advantage of the combined classifier ccomb over the text-only classifier ctext must arise from membership signals rather than distributional artifacts is not justified. MIA features are deterministic functions of x and f; when the suspect set (natural) and the synthetic held-out set (generated) differ in distribution, the MIA features encode that same shift, so ccomb can beat ctext on non-member data purely by exploiting distribution shift. Table 4 provides direct evidence: on non-member sets, AUC Comb exceeds AUC Text for FreeLaw (53.9 vs 51.4), Github (55.4 vs 53.9), EuroParl (53.9 vs 51.4), PubMed Central (55.5 vs 54.7), and StackExchange (55.0 vs 53.0). The corresponding p-values (0.09, 0.07, 0.07, 0.11, 0.06) do not establish a low false-positive rate; four of these five are below 0.1. The statement in Section 5.3 that p-values are "higher than 0.1 on all the non-member sets" is contradicted by StackExchange (0.06), FreeLaw (0.09), Github (0.07), and EuroParl (0.07). The authors should provide a control experiment with a genuinely in-distribution non-member held-out set, or otherwise demonstrate that the null distribution of their test is calibrated.
  2. [Section 5.5 and Eq. (2)] The paired t-test does not account for the fact that the two classifiers are trained once on the same training split and then evaluated on the same test pairs. The per-pair differences are therefore correlated rather than independent, likely inflating statistical significance. In addition, multiple test pairs derive from the same source document; Appendix C lists maximum snippets per document ranging from 5 to 2000, which introduces clustering by document. The description of the Sidak-corrected p-value aggregation across random seeds is too brief to be reproducible (number of seeds, how p-values are combined, and how ties are handled are unspecified). Exact p-values of 1.00 in many non-member rows of Table 4 are unexpected for a t-test and suggest an uncalibrated or degenerate test. A resampling-based test (e.g., bootstrapping at the document level, or a permutation test that refits the classifiers) would be more appropriate.
  3. [Appendix C and Section 4.1] The per-dataset choice of sequence length and maximum snippets per document is made by minimizing the AUC of the text classifier on the very suspect and synthetic data that are later used in the DI test. This constitutes tuning on the evaluation data: it can make the text classifier artificially weak, thereby biasing the comparison in favor of the combined classifier. The sensitivity analysis in Appendix K varies the number of epochs and the sample size, but not these two key generation hyperparameters. The authors should either provide a fixed protocol for selecting these values (e.g., from a separate development set or from the generator's validation loss) or demonstrate that the conclusions are robust to a range of these choices.
minor comments (6)
  1. [Section 5.3] The claim that p-values are "higher than 0.1 on all the non-member sets" is directly contradicted by Table 4; the sentence should be corrected to state the actual range and to discuss the borderline cases.
  2. [Figure 5 caption] The caption refers to "MeanDataset" but defines it as "the maximum p-value of all subsets"; this is likely a typo for "MaxDataset".
  3. [Section 4.1] The sentence "we 1 first segment" contains a stray "1" and should read "we first segment".
  4. [Section 4.3, Eq. (3)] The notation "wi∈R" should use mathbb, i.e., w_i \in \mathbb{R}; also, the equation formatting for expectations could be improved for readability.
  5. [Appendix A.1] The example in the text completion prompt contains a stray token "pneumoniae." that appears out of place and should be removed or clarified.
  6. [Tables 3 and 4] The AUC values are reported as point estimates without standard errors or confidence intervals; given the small differences (often 1–4 AUC points), reporting variability across the random seeds would help the reader judge effect sizes.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the DI decision is an empirical measurement validated against external membership labels, not a derivation from its own fitted inputs.

full rationale

The paper's inference target is the external target model's training membership, and its claimed novelty is the synthetic held-out generation plus a dual-classifier calibration. The DI decision is not derived by fitting a constant to the known membership labels: ctext and ccomb are trained on train splits of the suspect/held-out sets, evaluated on test splits, and the p-value is computed from the paired difference ccomb(xval)-ccomb(xsus) versus ctext(xval)-ctext(xsus) under the null hypothesis in Eq. (2). The outcome is then compared against independent ground-truth membership labels in Tables 3-5. The only candidate for a circular step is the Section 4.2 identification of any ccomb advantage over ctext with membership signal; that is an unproven causal assumption rather than an equation-level reduction, and it is not a fitted parameter renamed as a prediction. No load-bearing self-citation or imported uniqueness theorem is used: prior DI works are cited as background and as baselines, and the method is validated against external benchmarks including Pile subsets and OLMo with Dolma/Paloma data. The per-dataset snippet-length selection in Appendix C is a hyperparameter choice, not a fit of the target membership outcome. The strongest actual weakness is empirical, not circular: several non-member p-values in Table 4 (e.g., StackExchange 0.06, FreeLaw 0.09, Github 0.07) are close to the 0.05 threshold, and the paper's claim that all non-member p-values exceed 0.1 is contradicted by its own table; but this is a correctness and calibration concern, not a reduction of the prediction to its inputs. Therefore no significant circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new theoretical entities. It relies on a set of domain assumptions about the strength of membership signals and the validity of the t-test. The main free parameters are the snippet lengths and per-subset sampling choices, which are tuned to the data.

free parameters (3)
  • Sequence length (32 vs 64 tokens) = 32 or 64 depending on subset
    The authors choose 32 or 64 tokens per snippet for each Pile subset (Table A2). This choice is a free parameter that affects the distributional gap and membership signal, and it is tuned per dataset to achieve low text-classifier AUC. The paper acknowledges this in Appendix C: 'we suggest choosing the token numbers when the text-classifier has the minimal AUC'.
  • Max snippets per document = 5 to 200 depending on subset
    In Table A2, each Pile subset has a chosen maximum number of snippets per document to prevent over-sampling long documents. This is a per-dataset design choice.
  • LoRA rank and training epochs for generator = rank=32, epochs=100, lr=2e-4
    These are hyperparameters, but they are not fitted to the data in a problematic way; they are standard values. Still, they are free parameters of the method.
assumptions (3)
  • domain assumption The target model's MIA scores, computed on suffixes of length 32-64, carry a detectable membership signal after only one epoch of fine-tuning.
    This is the premise of the whole evaluation; if the signal is too weak, the method fails. The paper's experiments on 410M and 1B models show it is detectable in their settings, but it is not guaranteed for all real-world cases.
  • domain assumption The paired t-test on classifier output differences is valid, meaning the differences are approximately normal and the pairing is meaningful.
    The paper uses a t-test on the paired differences c_comb - c_text. No normality test or bootstrapping is reported, and the number of pairs (1000) is large enough for CLT, but the independence of pairs is questionable because they share prefixes.
  • domain assumption The text classifier is weak enough that its AUC is low on the synthetic held-out set, so the generation gap is small.
    The method's success depends on c_text having low AUC (around 52-56%); if a stronger classifier could separate real from synthetic, the method would fail. The paper uses a 2-layer GPT2 classifier and shows it has low AUC, but this is a choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unlocking Post-hoc Dataset Inference with Synthetic Data." pith.science (2026). https://pith.science/paper/VS6HEFFC

@misc{pith2026250615271,
  author       = {Pith},
  title        = {Pith review of: Unlocking Post-hoc Dataset Inference with Synthetic Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VS6HEFFC}},
  note         = {Machine review of arXiv:2506.15271}
}
read the original abstract

The remarkable capabilities of Large Language Models (LLMs) can be mainly attributed to their massive training datasets, which are often scraped from the internet without respecting data owners' intellectual property rights. Dataset Inference (DI) offers a potential remedy by identifying whether a suspect dataset was used in training, thereby enabling data owners to verify unauthorized use. However, existing DI methods require a private set-known to be absent from training-that closely matches the compromised dataset's distribution. Such in-distribution, held-out data is rarely available in practice, severely limiting the applicability of DI. In this work, we address this challenge by synthetically generating the required held-out set. Our approach tackles two key obstacles: (1) creating high-quality, diverse synthetic data that accurately reflects the original distribution, which we achieve via a data generator trained on a carefully designed suffix-based completion task, and (2) bridging likelihood gaps between real and synthetic data, which is realized through post-hoc calibration. Extensive experiments on diverse text datasets show that using our generated data as a held-out set enables DI to detect the original training sets with high confidence, while maintaining a low false positive rate. This result empowers copyright owners to make legitimate claims on data usage and demonstrates our method's reliability for real-world litigations. Our code is available at https://github.com/sprintml/PostHocDatasetInference.

Figures

Figures reproduced from arXiv: 2506.15271 by the authors.

Figure 1
Figure 1. Dataset Inference Procedure with Synthetic Held-Out Data. This figure presents a high-level overview of how the proposed dataset inference (DI) process will take place in real-world use cases. (a-d) LLM providers scrape proprietary author data from the internet, and train an LLM on it. Authors who suspect unauthorized use may approach an arbiter with a copyright claim. To resolve such a dispute, the arbiter must per… view at source ↗
Figure 2
Figure 2. Probability Distribution Function (PDF) of target model perplexities. We show the comparison between (a) the member and held-out, and (b) the non-member and held-out sets. behind this observation is that each blog has different con￾tent and topics, which brings different words across the non￾member and held-out documents. The gap is enlarged when we sample more sequences from each blog post. Conse￾quently, this dist… view at source ↗
Figure 3
Figure 3. Held-out Data Generation (Left Panel): (1) The suspect dataset is first segmented into text snippets. (2) These snippets are shuffled and split into a generator training set and an inference set. (3) A generator model is trained on the suspect dataset using a suffix completion task. (4) The trained generator produces synthetic held-out data that mimics the suspect set. Post-hoc Calibration (Right Panel): (1) The sus… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Weight Constraint. The weights for the MIA and text scores are constrained to (0,1) with Sigmoid function. two sets ydiff as follows: ydiff = E[ Xn i=1 wiMIAi(xval)] − E[ Xn i=1 wiMIAi(xsus)] = Xn i=1 wi(E[MIAi(xval)] − E[MIAi(xsus)]) > 0 if Dsus is member set, otherwi…
Figure 5
Figure 5. Figure 5: The p-values of member sets with change in sample size. MedianDataset denotes the median p-value of different datasets, and MeanDataset is the maximum p-value of all subsets. Number of samples refers to the total size of both suspect and validation sets. the above sett…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 28 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    use of copyrighted work https://www.nytimes.com/2023/12/27/business/media/new-york-times-open-ai-microsoft-lawsuit.html

    The times sues openai and microsoft over a.i. use of copyrighted work https://www.nytimes.com/2023/12/27/business/media/new-york-times-open-ai-microsoft-lawsuit.html. 2023. URL https://www.nytimes.com/2023/12/27/business/media/new-york-times-open-ai-microsoft-lawsuit.html

    work page 2023

  3. [3]

    Sarah silverman and authors sue openai and meta over copyright infringement. 2023. URL https://www.nytimes.com/2023/07/10/arts/sarah-silverman-lawsuit-openai-meta.html

    work page 2023

  4. [4]

    Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source llms

    Balloccu, S., Schmidtov \'a , P., Lango, M., and Du s ek, O. Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source llms. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 67--93, 2024

    work page 2024

  5. [5]

    G., Bradley, H., O’Brien, K., Hallahan, E., Khan, M

    Biderman, S., Schoelkopf, H., Anthony, Q. G., Bradley, H., O’Brien, K., Hallahan, E., Khan, M. A., Purohit, S., Prashanth, U. S., Raff, E., et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp.\ 2397--2430. PMLR, 2023

    2023

  6. [6]

    Extracting training data from large language models

    Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp.\ 2633--2650, 2021

    2021

  7. [7]

    Blind baselines beat membership inference attacks for foundation models

    Das, D., Zhang, J., and Tram \`e r, F. Blind baselines beat membership inference attacks for foundation models. arXiv preprint arXiv:2406.16201, 2024

    arXiv 2024

  8. [8]

    Flocks of stochastic parrots: Differentially private prompt learning for large language models

    Duan, H., Dziedzic, A., Papernot, N., and Boenisch, F. Flocks of stochastic parrots: Differentially private prompt learning for large language models. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023 a

    work page 2023

Show all 47 references
  1. [9]

    On the privacy risk of in-context learning

    Duan, H., Dziedzic, A., Yaghini, M., Papernot, N., and Boenisch, F. On the privacy risk of in-context learning. In The 61st Annual Meeting Of The Association For Computational Linguistics, 2023 b

    2023

  2. [10]

    Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841, 2024

    Duan, M., Suri, A., Mireshghallah, N., Min, S., Shi, W., Zettlemoyer, L., Tsvetkov, Y., Choi, Y., Evans, D., and Hajishirzi, H. Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841, 2024

    2024 arXiv

  3. [11]

    V., Zhao, X., Oliveira, A

    Duarte, A. V., Zhao, X., Oliveira, A. L., and Li, L. De-cop: Detecting copyrighted content in language models training data. arXiv preprint arXiv:2402.09910, 2024

    2024 arXiv

  4. [12]

    The llama 3 herd of models

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  5. [13]

    CDI: Copyrighted Data Identification in Diffusion Models

    Dubiński, J., Kowalczuk, A., Boenisch, F., and Dziedzic, A. CDI: Copyrighted Data Identification in Diffusion Models . In The IEEE CVF Computer Vision and Pattern Recognition Conference (CVPR), 2025

    2025

  6. [14]

    A., Guan, J., and Papernot, N

    Dziedzic, A., Dhawan, N., Kaleem, M. A., Guan, J., and Papernot, N. On the difficulty of defending self-supervised learning against model extraction. In ICML (International Conference on Machine Learning), 2022 a

    2022

  7. [15]

    A., Dhawan, N., Guan, J., Cattan, Y., Boenisch, F., and Papernot, N

    Dziedzic, A., Duan, H., Kaleem, M. A., Dhawan, N., Guan, J., Cattan, Y., Boenisch, F., and Papernot, N. Dataset inference for self-supervised models. Advances in Neural Information Processing Systems, 35: 0 12058--12070, 2022 b

    2022

  8. [16]

    Membership inference attacks against fine-tuned large language models via self-prompt calibration

    Fu, W., Wang, H., Gao, C., Liu, G., Li, Y., and Jiang, T. Membership inference attacks against fine-tuned large language models via self-prompt calibration. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  9. [17]

    The pile: An 800gb dataset of diverse text for language modeling

    Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020

    2020 arXiv

  10. [18]

    and Surdeanu, M

    Golchin, S. and Surdeanu, M. Time travel in llms: Tracing data contamination in large language models. arXiv preprint arXiv:2308.08493, 2023

    2023 arXiv

  11. [19]

    Olmo: Accelerating the science of language models

    Groeneveld, D., Beltagy, I., Walsh, E., Bhagia, A., Kinney, R., Tafjord, O., Jha, A., Ivison, H., Magnusson, I., Wang, Y., et al. Olmo: Accelerating the science of language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu...

    2024

  12. [20]

    E., Backes, M., and Dziedzic, A

    Hanke, V., Blanchard, T., Boenisch, F., Olatunji, I. E., Backes, M., and Dziedzic, A. Open llms are necessary for current private adaptations and outperform their closed alternatives. In Thirty-Eighth Conference on Neural Information Processing Systems (NeurIPS), 2024

    2024

  13. [21]

    A., Jagielski, M., Kaissis, G., Lee, K., Nasr, M., Ghalebikesabi, S., Mireshghallah, N., Annamalai, M

    Hayes, J., Shumailov, I., Choquette-Choo, C. A., Jagielski, M., Kaissis, G., Lee, K., Nasr, M., Ghalebikesabi, S., Mireshghallah, N., Annamalai, M. S. M. S., Shilov, I., Meeus, M., de Montjoye, Y.-A., Boenisch, F., Dziedzic, A., and Cooper, A. F. Strong membership inference at...

    2025

  14. [22]

    PANORAMIA : Privacy auditing of machine learning models without retraining

    Kazmi, M., Lautraite, H., Akbari, A., Tang, Q., Soroco, M., Wang, T., Gambs, S., and L \'e cuyer, M. PANORAMIA : Privacy auditing of machine learning models without retraining. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  15. [23]

    Privacy attacks on image autoregressive models

    Kowalczuk, A., Dubiński, J., Boenisch, F., and Dziedzic, A. Privacy attacks on image autoregressive models. In Forty-Second International Conference on Machine Learning (ICML), 2025

    2025

  16. [24]

    and Flanigan, J

    Li, C. and Flanigan, J. Task contamination: Language models may not be few-shot anymore. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 18471--18480, 2024

    2024

  17. [25]

    H., Tafjord, O., Schwenk, D., Walsh, E., Elazar, Y., Lo, K., et al

    Magnusson, I., Bhagia, A., Hofmann, V., Soldaini, L., Jha, A. H., Tafjord, O., Schwenk, D., Walsh, E., Elazar, Y., Lo, K., et al. Paloma: A benchmark for evaluating language model fit. Advances in Neural Information Processing Systems, 37: 0 64338--64376, 2024

    2024

  18. [26]

    and Suri, A

    Maini, P. and Suri, A. Reassessing emnlp 2024’s best paper: Does divergence-based calibration for membership inference attacks hold up? 2024. URL https://www.anshumansuri.com/blog/2024/calibrated-mia/. Accessed January 29, 2025

    2024

  19. [27]

    Dataset inference: Ownership resolution in machine learning

    Maini, P., Yaghini, M., and Papernot, N. Dataset inference: Ownership resolution in machine learning. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021

    2021

  20. [28]

    LLM dataset inference: Did you train on my dataset? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    Maini, P., Jia, H., Papernot, N., and Dziedzic, A. LLM dataset inference: Did you train on my dataset? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  21. [29]

    Membership inference attacks against language models via neighbourhood comparison

    Mattern, J., Mireshghallah, F., Jin, Z., Schoelkopf, B., Sachan, M., and Berg-Kirkpatrick, T. Membership inference attacks against language models via neighbourhood comparison. In Findings of the Association for Computational Linguistics: ACL 2023, pp.\ 11330--11343, 2023

    2023

  22. [30]

    Sok: Membership inference attacks on llms are rushing nowhere (and how to fix it)

    Meeus, M., Shilov, I., Jain, S., Faysse, M., Rei, M., and de Montjoye, Y.-A. Sok: Membership inference attacks on llms are rushing nowhere (and how to fix it). arXiv preprint arXiv:2406.17975, 2024

    2024 arXiv

  23. [31]

    Sim PO : Simple preference optimization with a reference-free reward

    Meng, Y., Xia, M., and Chen, D. Sim PO : Simple preference optimization with a reference-free reward. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  24. [32]

    S., Ladhak, F., and Hashimoto, T

    Oren, Y., Meister, N., Chatterji, N. S., Ladhak, F., and Hashimoto, T. Proving test set contamination in black-box language models. In The Twelfth International Conference on Learning Representations, 2024

    2024

  25. [33]

    The fineweb datasets: Decanting the web for the finest text data at scale

    Penedo, G., Kydl \' c ek, H., Lozhkov, A., Mitchell, M., Raffel, C., Von Werra, L., Wolf, T., et al. The fineweb datasets: Decanting the web for the finest text data at scale. arXiv preprint arXiv:2406.17557, 2024

    2024 arXiv

  26. [34]

    D., Ermon, S., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  27. [35]

    and Santacana, E

    Rahman, N. and Santacana, E. Beyond fair use: Legal risk evaluation for training llms on copyrighted text. 2023. URL https://genlaw.org/CameraReady/57.pdf

    2023

  28. [36]

    Getty images lawsuit says stability ai misused photos to train AI , 2023

    Reuters. Getty images lawsuit says stability ai misused photos to train AI , 2023. URL https://www.reuters.com/legal/getty-images-lawsuit-says-stability-ai-misused-photos-train-ai-2023-02-06/

    2023

  29. [37]

    To the cutoff

    Roberts, M., Thakur, H., Herlihy, C., White, C., and Dooley, S. To the cutoff... and beyond? a longitudinal perspective on llm data contamination. In The Twelfth International Conference on Learning Representations, 2024

    2024

  30. [38]

    Detecting pretraining data from large language models

    Shi, W., Ajith, A., Xia, M., Huang, Y., Liu, D., Blevins, T., Chen, D., and Zettlemoyer, L. Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations, 2024

    2024

  31. [39]

    Membership inference attacks against machine learning models

    Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp.\ 3--18. IEEE, 2017

    2017

  32. [40]

    Rectangular confidence regions for the means of multivariate normal distributions

    S id \'a k, Z. Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American statistical association, 62 0 (318): 0 626--633, 1967

    1967

  33. [41]

    Dolma: an open corpus of three trillion tokens for language model pretraining research

    Soldaini, L., Kinney, R., Bhagia, A., Schwenk, D., Atkinson, D., Authur, R., Bogin, B., Chandu, K., Dumas, J., Elazar, Y., et al. Dolma: an open corpus of three trillion tokens for language model pretraining research. In Proceedings of the 62nd Annual Meeting of the Associatio...

    2024

  34. [42]

    Weber, M., Fu, D. Y., Anthony, Q., Oren, Y., Adams, S., Alexandrov, A., Lyu, X., Nguyen, H., Yao, X., Adams, V., Athiwaratkun, B., Chalamala, R., Chen, K., Ryabinin, M., Dao, T., Liang, P., Ré, C., Rish, I., and Zhang, C. Redpajama: an open dataset for training large language ...

    2024

  35. [43]

    Unveiling security, privacy, and ethical concerns of chatgpt

    Wu, X., Duan, R., and Ni, J. Unveiling security, privacy, and ethical concerns of chatgpt. Journal of Information and Intelligence, 2023

    2023

  36. [44]

    V., Murray, K., and Kim, Y

    Xu, H., Sharaf, A., Chen, Y., Tan, W., Shen, L., Durme, B. V., Murray, K., and Kim, Y. J. Contrastive preference optimization: Pushing the boundaries of LLM performance in machine translation. In Forty-first International Conference on Machine Learning, 2024. URL https://openr...

    2024

  37. [45]

    Privacy risk in machine learning: Analyzing the connection to overfitting

    Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pp.\ 268--282. IEEE, 2018

    2018

  38. [46]

    Membership inference attacks cannot prove that a model was trained on your data

    Zhang, J., Das, D., Kamath, G., and Tram \`e r, F. Membership inference attacks cannot prove that a model was trained on your data. arXiv preprint arXiv:2409.19798, 2024 a

    2024 arXiv

  39. [47]

    F., and Li, H

    Zhang, J., Sun, J., Yeats, E., Ouyang, Y., Kuo, M., Zhang, J., Yang, H. F., and Li, H. Min-k\ arXiv preprint arXiv:2404.02936, 2024 b

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.