Pith. sign in

REVIEW 10 cited by

Dataset Inference: Ownership Resolution in Machine Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2104.10706 v1 pith:PIYCCXLD submitted 2021-04-21 stat.ML cs.CRcs.LG

Pratyush Maini , Mohammad Yaghini , Nicolas Papernot This is my paper · ORCID
classification stat.MLcs.CRcs.LG
keywords modeldatasettraininginferencestolenadversarydataknowledge
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

With increasingly more data and computation involved in their training, machine learning models constitute valuable intellectual property. This has spurred interest in model stealing, which is made more practical by advances in learning with partial, little, or no supervision. Existing defenses focus on inserting unique watermarks in a model's decision surface, but this is insufficient: the watermarks are not sampled from the training distribution and thus are not always preserved during model stealing. In this paper, we make the key observation that knowledge contained in the stolen model's training set is what is common to all stolen copies. The adversary's goal, irrespective of the attack employed, is always to extract this knowledge or its by-products. This gives the original model's owner a strong advantage over the adversary: model owners have access to the original training data. We thus introduce $dataset$ $inference$, the process of identifying whether a suspected model copy has private knowledge from the original model's dataset, as a defense against model stealing. We develop an approach for dataset inference that combines statistical testing with the ability to estimate the distance of multiple data points to the decision boundary. Our experiments on CIFAR10, SVHN, CIFAR100 and ImageNet show that model owners can claim with confidence greater than 99% that their model (or dataset as a matter of fact) was stolen, despite only exposing 50 of the stolen model's training points. Dataset inference defends against state-of-the-art attacks even when the adversary is adaptive. Unlike prior work, it does not require retraining or overfitting the defended model.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dataset Ownership Verification for Pre-trained Masked Models

    cs.CV 2025-07 conditional novelty 6.0 of 10

    DOV4MM detects whether a masked pre-trained model was trained on a given dataset via relative embedding reconstruction difficulty, reporting p<0.05 in tests on ImageNet-1K and WikiText-103.

  2. Suitability Filter: A Statistical Framework for Classifier Evaluation in Real-World Deployment Settings

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A statistical non-inferiority test on estimated per-sample correctness probabilities flags when a classifier's accuracy on unlabeled user data drops by more than a chosen margin relative to its test set.

  3. Has My System Prompt Been Used? Large Language Model Prompt Membership Inference

    cs.AI 2025-02 conditional novelty 6.0 of 10

    A permutation test on BERT embeddings of LLM outputs can detect, with statistical significance, when response distributions differ because a chat service uses a different system prompt than a candidate prompt.

  4. Dataset Ownership Verification in Contrastive Pre-trained Models

    cs.LG 2025-02 conditional novelty 6.0 of 10

    The authors define a contrastive relationship gap and use a one-tailed T-test to detect whether a suspect encoder was pre-trained on a target dataset.

  5. Membership Inference Attacks on Tokenizers of Large Language Models

    cs.CR 2025-10 conditional novelty 5.0 of 10

    Tokenizers leak dataset membership: using distinctive tokens that appear in a dataset's text, an attacker can detect with AUC up to 0.77 whether that dataset was part of tokenizer training.

  6. MISLEADER: Defending against Model Extraction with Ensembles of Distilled Models

    cs.CR 2025-06 reject novelty 5.0 of 10

    MISLEADER trains an ensemble of distilled models that stay accurate for benign users but output misleading predictions on augmented inputs, reducing clone model accuracy in model extraction attacks.

  7. Quality over Quantity: Boosting Data Efficiency Through Ensembled Multimodal Data Curation

    cs.LG 2025-02 conditional novelty 5.0 of 10

    EcoDatum filters web image-text data by ensembling eight unimodal and multimodal quality scorers with weak-supervision weighting, reporting a DataComp small-scale average score of 0.182.

  8. A Systematic Survey of Model Extraction Attacks and Defenses: State-of-the-Art and Perspectives

    cs.CR 2025-08 conditional novelty 4.0 of 10

    The paper classifies model extraction attacks and defenses into attack, defense, and computing environment categories and surveys their current state.

  9. Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models

    cs.CL 2025-08 unverdicted novelty 4.0 of 10

    Fine-tuning LLMs on repeated sensitive data is claimed to raise privacy leakage to 60-75%, and four filters are said to cut leakage to 0% while keeping 94.7% of utility.

  10. Dataset Ownership in the Era of Large Language Models

    cs.CR 2025-09 conditional novelty 2.0 of 10

    A survey that categorizes dataset copyright protection into non-intrusive, minimally-intrusive, and maximally-intrusive methods.

Pith tools