Pith. sign in

REVIEW 3 major objections 4 minor 62 references

Dataset Ownership Verification for Pre-trained Masked Models

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DOV4MM verifies whether a black-box masked pre-trained model was trained on a given public dataset by comparing masked-information reconstruction difficulty for seen versus unseen samples.

desk verdict First DOV method for masked models, but the test statistic secretly compares private samples against the defender's holdout, so the claimed verification scope is narrower than the paper says. read the letter →

arxiv 2507.12022 v1 pith:6FUEHWXA submitted 2025-07-16 cs.CV

Yuechen Xie , Jie Song , Yicheng Shan , Xiaoyan Zhang , Yuanyu Wan , Shengxuming Zhang , Jiarui Duan , Mingli Song This is my paper
classification cs.CV
keywords datasetownershipverificationmaskedmodelingpre-trainedmodelsembeddingreconstructiondifficultymembershipinferenceblack-boxself-supervisedlearninghypothesistesting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a dataset owner can determine whether a suspicious black-box masked model was pre-trained on the owner's public dataset, without watermarking the data and without seeing the model's internals. The proposed DOV4MM trains a small decoder on the suspect model's own embeddings, then measures the relative embedding reconstruction difficulty — how much harder masked information is to rebuild for private unseen samples than for the public dataset's own validation samples — and feeds that measure into a one-tailed paired t-test. A p-value below 0.05 says the model was trained on the protected dataset; a p-value above 0.05 says it was not. The authors report this verdict is correct across ten masked image models on ImageNet-1K and four masked language models on WikiText-103, using only about 3% of the protected data, in settings where prior watermark-based and inference-based verification approaches fail or do not transfer.

What carries the argument

The load-bearing object is the relative embedding reconstruction difficulty $\Delta R$: the per-iteration gap between two embedding reconstruction difficulties measured against a common reference set. The atomic quantity is the per-sample embedding reconstruction difficulty $R(x, t, \hat{t}, M, M_d) = \frac{\|(M_d(e_t) - e) \odot (1 - \hat{t})\|_2^2}{\|1 - \hat{t}\|_1}$, the squared reconstruction error at masked positions in the suspect model's embedding space after a decoder $M_d$ trained on that model's own outputs tries to fill them in. DOV4MM computes $\Delta R$ for the public validation split and for the defender's private set, both relative to the training split, over $K$ sampling iterations, and submits the paired differences to a one-tailed pairwise t-test; the test's p-value is the decision statistic that converts the raw reconstruction gap into an ownership verdict.

What would settle it

Pre-train a masked model on a partial mixture of the protected dataset and an unrelated one — for instance, half of $D_{pub}$ interleaved with an equal-sized foreign dataset — run DOV4MM with $D_{pub}$ as the defended dataset, and check whether the p-value still falls below 0.05; the paper reports only the fully overlapping illegal case and fully disjoint legal cases, so a partial-overlap result that exonerates a model that did train on $D_{pub}$ would falsify the central claim. A complementary probe is to train the same architecture with strong weight decay or explicit anti-memorization regularization and watch whether the $\Delta R_{pt} - \Delta R_{vt}$ gap collapses below the significance threshold.

Watch

Extended reading notes

Core claim

The paper's central claim is that pre-trained masked models retain a measurable memory of their training data: in the model's embedding space, masked information from samples seen during pre-training is easier to reconstruct than masked information from samples never seen (Observation 1). On that basis DOV4MM trains a decoder $M_d$ on the suspect model's own embeddings, computes the embedding reconstruction difficulty of masked positions for three sample groups — the public training subset, the public validation subset, and a private set the model never saw — and forms the relative embedding reconstruction difficulties $\Delta R_{vt}$ and $\Delta R_{pt}$ against the common training reference. A one-tailed paired t-test on these paired values decides ownership: rejecting the null hypothesis (p < 0.05) marks the suspect illegal; failing to reject marks it legal. The paper reports this test succeeding across ten masked image modeling methods and four masked language modeling methods, and surviving early stopping, downstream fine-tuning, masked-modeling fine-tuning, and an adaptive loss that explicitly tries to equalize seen and unseen reconstruction difficulty.

Load-bearing premise

The entire method rests on Observation 1, the empirically asserted claim that masked information is easier to reconstruct in the embedding space of samples a model saw during pre-training than of samples it never saw; the paper gives no proof of this gap, and if it shrinks or disappears under strong regularization, heavy fine-tuning, or partial overlap between the protected dataset and the model's real training data, the p-value test loses its power to separate illegal from legal models.

Editorial extensions

If this is right

  • Dataset owners can audit a suspect masked model through its output API alone, without altering or watermarking their data, which removes the main drawback of backdoor-based verification.
  • Verification is cheap enough for routine use: on ImageNet-1K, DOV4MM needs roughly 3% of the protected dataset (about 40,000 images) and about 353 seconds on a single GPU, whereas the dataset-inference baseline must process the entire dataset.
  • The test transfers across mask-based pre-training families — ten masked image models and four masked language models — indicating the seen-versus-unseen reconstruction gap is a general property of masked pre-training rather than an artifact of one architecture or modality.
  • Tested evasion attempts fail to break it: early stopping, downstream fine-tuning, fine-tuning with masked modeling, and a custom adaptive loss that tries to equalize reconstruction difficulty all leave the p-value below 0.05.
  • Because the check needs only the defender's own data and the model's feature outputs, it can be applied after the fact to models already trained or deployed, not only to models whose training a defender can influence through watermarks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Observation 1 is a memorization signal, its strength should track how hard the model overfit during pre-training; this predicts the test will become harder to pass on very large or highly regularized pre-training runs, a regime the paper does not examine.
  • The decoder is trained on the suspect's own embeddings, so the verdict fingerprints the model's representational geometry as much as the data itself; a suspect who reproduces the protected data with a different architecture or through distillation may weaken the signal, which the paper does not test.
  • The t-test framing invites a stronger adversary than the one studied: a suspect who knows the exact $\Delta R$ statistic could pre-train directly against it, for example by regularizing the seen-unseen gap at precisely the masked positions the decoder probes, rather than with the generic equalization loss the paper tries.
  • The same private probe set and protocol could serve as a standing license-monitoring service — one decoder per suspect model, one probe set per dataset — since verifying a new suspect costs only decoder training plus feature queries, a deployment pattern the authors leave implicit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes DOV4MM, a black-box dataset ownership verification method for pre-trained masked models. The defender splits a public dataset into a training subset Dt and a validation subset Dv, trains a light decoder on Dt to reconstruct masked embeddings of the suspicious encoder, and computes per-iteration reconstruction difficulties for Dt, Dv, and a private set Dpvt. A one-sided paired t-test is then applied to the relative difficulties ΔRvt and ΔRpt to decide whether the suspicious model was pre-trained on the public dataset. The method is evaluated on ImageNet subsets and ImageNet-1K with ten masked image modeling methods, and on WikiText-103 subsets with four masked language models, claiming significant rejection for illegal cases and non-rejection for legal cases, with ablations and interference-resistance experiments.

Significance. If the central claim holds, DOV4MM is a useful addition to dataset protection: it is the first verification method tailored to masked models, works in a black-box setting, does not rely on backdoor watermarks, and the experimental scope is broad (ten MIM methods, four MLM methods, multiple architectures, fine-tuning, early stopping, and an adaptive attack). The authors also provide code and report detailed p-values in the supplementary material, which strengthens reproducibility. The significance is tempered by the fact that, as shown below, the tested statistic reduces to a comparison of the defender's holdout split against the private split, so the scope of the verified claim is narrower than the abstract states.

major comments (3)
  1. [§3.3, Definition 2, and Supplement Eq. (4)] The test statistic collapses to a direct comparison between the private set and the validation split, so the decoder baseline Dt cancels out. In the supplement, Eq. (4) defines d_k = ΔR^k_pt − ΔR^k_vt; substituting Definition 2 gives d_k = (R^k_p − R^k_t) − (R^k_v − R^k_t) = R^k_p − R^k_v. Thus the test rejects H0 when the private samples are harder to reconstruct than the specific validation samples Dv, not when the model has used Dpub as a whole. In every illegal experiment, Ms was pre-trained on the whole public dataset, so Dv was always a subset of the pre-training data. If a suspect pre-trained on a subset of Dpub that does not include Dv (for example, only on Dt or on an arbitrary 80% that misses Dv), the test would return p > 0.05 despite the use of data from Dpub. No partial-overlap experiment is reported in §4 or the supplement. The abstract's claim that DOV4MM ascertains whether Ms was 'pre-trained on a particular unlabeled dataset' is therefore supported only for the special case of full overlap with the defender's random holdout. The authors should either narrow the claim to verification of the entire dataset (or of the holdout split) or provide evidence, theoretical or experimental, that partial overlap with Dpub is detected.
  2. [Supplement §A.4, Algorithm 1] Algorithm 1 calls scipy.stats.ttest_ind(ΔRpt, ΔRvt), which is an independent-samples t-test, while the surrounding text and Eqs. (4)–(6) describe a paired one-sided t-test on the K paired differences. The paired and independent tests can give materially different p-values when the paired observations are correlated, which is the very situation here because both quantities are computed on the same model and the same random masks. If the experiments used Algorithm 1, then the reported p-values are not from the stated paired test; if the experiments used the paired formulation, then Algorithm 1 and the corresponding code are misleading. This is load-bearing for the statistical claim, and the authors should report which procedure was actually run and update the code/text accordingly.
  3. [§3.2.1, Observation 1] Observation 1, that the reconstruction difficulty of masked information is lower for seen samples than unseen samples in the embedding space of masked models, is the load-bearing empirical premise of the entire method. It is supported by the reported experiments, but it is not characterized in scope: there is no analysis of when the gap can shrink or disappear, for example under strong regularization, heavy fine-tuning with large learning rates, or adversarial training that explicitly minimizes the seen/unseen gap. The early-stopping, fine-tuning, and adaptive-attack experiments are helpful, but they do not establish a boundary of validity. Given that the method's decision rule and all conclusions depend on this observation, the paper should state it as a testable hypothesis and report conditions under which it fails, rather than presenting it as a general invariant.
minor comments (4)
  1. [Figure 3 caption and supplementary Figure 5] The caption of Figure 3 and the supplementary Figure 5 contain long runs of garbled glyph tokens (e.g., '/uni0000002c/uni00000031/...'); these should be cleaned to the intended text.
  2. [Supplement §A.3] In the MLM training settings, 'a learning rate of 5e-' is missing the exponent; it should read, for example, '5e-5'.
  3. [Algorithm 1] Line 3 of Algorithm 1 reads 'scipy.state.ttest ind(...)', which should be 'scipy.stats.ttest_ind(...)'.
  4. [§4.6, Table 5] The fine-tuning experiment on logits replaces the embedding mask with no mask at all and uses a three-layer fully connected decoder; this variant should be described more precisely, since it is no longer the same reconstruction-difficulty definition as in Eq. (3).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DOV4MM is an empirical membership-style test; the paired-test cancellation of Rt is a simplification, not a circular reduction.

full rationale

DOV4MM's derivation chain is empirical and self-contained; I find no step in which a prediction is equivalent to its inputs by construction. The decoder is trained only on Dt, the p-value is computed from held-out private samples Dpvt and the validation split Dv, no decision threshold is fitted to the target labels, and all hyperparameters are fixed and ablated rather than optimized on the test set. Observation 1 (seen samples reconstruct more easily) is an empirical premise tested against official pre-trained masked models and external baselines (DI4SSL, CTRL, PartCrop); it is not imported from a self-citation. The self-citations ([51], [52]) are contextual and not load-bearing. Algebraically, the paired test statistic in Eq. (4) reduces to Rp - Rv because the Rt baseline cancels, so the 'relative' reference does no work in the paired comparison; this is a simplification and a scope limitation (the test detects memorization of the particular split Dv rather than full-Dpub membership), but it is not a circular reduction: the claim is not defined in terms of the test outcome, and the experiments are consistent with the reduced statistic. No circularity is present.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central claim rests on one empirical observation about reconstruction difficulty, a set of hand-chosen hyperparameters that are ablated, and standard statistical assumptions. No new physical or conceptual entities are introduced.

free parameters (6)
  • K (sampling iterations) = 30
    Number of paired samples in the t-test; hand-chosen and ablated in Table 3f.
  • N (samples per iteration) = 1024
    Batch size for computing mean reconstruction difficulty; ablated in Table 3g.
  • Masking ratio = 75% (images), 20% (text)
    Fraction of input masked when computing reconstruction difficulty; ablated in Table 3e.
  • Decoder training set size |Dt| = 20,000
    Size of the public dataset subset used to train the decoder; ablated in Table 3d.
  • Decoder architecture = Transformer, depth 8, width 512, heads 16
    Chosen architecture; ablated in Tables 3a-c and found robust.
  • Significance level alpha = 0.05
    Threshold for rejecting the null hypothesis; standard, not fitted.
assumptions (3)
  • domain assumption Observation 1: In the embedding space of pre-trained masked models, reconstruction difficulty of masked information is lower for seen samples than unseen samples.
    This empirical claim is the foundation of DOV4MM; it is asserted from intuition and not proven. The method's success depends on this gap persisting across models and datasets.
  • standard math The paired differences used in the t-test are independent and approximately normally distributed.
    The one-tailed paired t-test requires these assumptions; K=30 is small and the differences may be correlated because they share the same baseline Dt, but the paper does not discuss this.
  • domain assumption The defender's private dataset Dpvt is drawn from the same distribution as Dpub and was not used in Ms's pre-training.
    This is required for the comparison between Rv and Rp to isolate the effect of memorization rather than domain shift.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dataset Ownership Verification for Pre-trained Masked Models." pith.science (2026). https://pith.science/paper/6FUEHWXA

@misc{pith2026250712022,
  author       = {Pith},
  title        = {Pith review of: Dataset Ownership Verification for Pre-trained Masked Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6FUEHWXA}},
  note         = {Machine review of arXiv:2507.12022}
}
abstract

High-quality open-source datasets have emerged as a pivotal catalyst driving the swift advancement of deep learning, while facing the looming threat of potential exploitation. Protecting these datasets is of paramount importance for the interests of their owners. The verification of dataset ownership has evolved into a crucial approach in this domain; however, existing verification techniques are predominantly tailored to supervised models and contrastive pre-trained models, rendering them ill-suited for direct application to the increasingly prevalent masked models. In this work, we introduce the inaugural methodology addressing this critical, yet unresolved challenge, termed Dataset Ownership Verification for Masked Modeling (DOV4MM). The central objective is to ascertain whether a suspicious black-box model has been pre-trained on a particular unlabeled dataset, thereby assisting dataset owners in safeguarding their rights. DOV4MM is grounded in our empirical observation that when a model is pre-trained on the target dataset, the difficulty of reconstructing masked information within the embedding space exhibits a marked contrast to models not pre-trained on that dataset. We validated the efficacy of DOV4MM through ten masked image models on ImageNet-1K and four masked language models on WikiText-103. The results demonstrate that DOV4MM rejects the null hypothesis, with a $p$-value considerably below 0.05, surpassing all prior approaches. Code is available at https://github.com/xieyc99/DOV4MM.

Figures

Figures reproduced from arXiv: 2507.12022 by the authors.

Figure 1
Figure 1. The overview of DOV4MM’s motivation. In the embed [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overview of DOV4MM (best viewed under color conditions). [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results of four methods on ImageNet-50 (the first line) and ImageNet-100 (the second line). The pre-training dataset of the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The results on WikiText-103 subsets. Ms is pre-trained on WikiText-103-50k (left) and WikiText-103-100k (right) respec￾tively. On the x-axis, “Wiki-50k” and “Wiki-100k” represent WikiText-103-50k and WikiText-103-100k, respectively. The re￾maining identifiers are the s…
Figure 5
Figure 5. Figure 5: The distribution of relative embedding reconstruction [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 43 canonical work pages

  1. [1]

    Beit: Bert pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021. 2, 5

    arXiv 2021

  2. [2]

    Food-101–mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, pro- ceedings, part VI 13, pages 446–461. Springer, 2014. 5, 1

    work page 2014

  3. [3]

    Poisoning and backdooring contrastive learning

    Nicholas Carlini and Andreas Terzis. Poisoning and backdooring contrastive learning. arXiv preprint arXiv:2106.09667, 2021. 2

    arXiv 2021

  4. [4]

    Membership inference attacks from first principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897–1914. IEEE, 2022. 3

    work page 2022

  5. [5]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 2, 3

    work page 2020

  6. [6]

    Exploring simple siamese rep- resentation learning

    Xinlei Chen and Kaiming He. Exploring simple siamese rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 15750–15758, 2021

    work page 2021

  7. [7]

    An empirical study of training self-supervised vision transformers

    Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9640–9649, 2021. 2, 3

    work page 2021

  8. [8]

    Context autoencoder for self- supervised representation learning

    Xiaokang Chen, Mingyu Ding, Xiaodi Wang, Ying Xin, Shentong Mo, Yunhao Wang, Shumin Han, Ping Luo, Gang Zeng, and Jingdong Wang. Context autoencoder for self- supervised representation learning. International Journal of Computer Vision, 132(1):208–223, 2024. 5

    work page 2024

Show all 62 references
  1. [9]

    Label-only membership infer- ence attacks

    Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label-only membership infer- ence attacks. In International conference on machine learn- ing, pages 1964–1974. PMLR, 2021. 3

    1964

  2. [10]

    Unsupervised cross-lingual representation learning at scale

    A Conneau. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116 , 2019. 3, 8

    1911 arXiv

  3. [11]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 1, 3, 5

    2009

  4. [12]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 2, 3, 8

    2018 arXiv

  5. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 3, 5

    2010 arXiv

  6. [14]

    Dataset inference for self- supervised models

    Adam Dziedzic, Haonan Duan, Muhammad Ahmad Kaleem, Nikita Dhawan, Jonas Guan, Yannis Cattan, Franziska Boenisch, and Nicolas Papernot. Dataset inference for self- supervised models. Advances in Neural Information Pro- cessing Systems, 35:12058–12070, 2022. 3, 6

    2022

  7. [15]

    Eva: Exploring the limits of masked visual representa- tion learning at scale

    Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva: Exploring the limits of masked visual representa- tion learning at scale. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition,...

    2023

  8. [16]

    The Pile: An 800GB dataset of diverse text for language modeling

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, An- ish Thite, Noa Nabeshima, et al. The Pile: An 800GB dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. 1

    2020 arXiv

  9. [17]

    Domain watermark: Effec- tive and harmless dataset copyright protection is closed at hand

    Junfeng Guo, Yiming Li, Lixu Wang, Shu-Tao Xia, Heng Huang, Cong Liu, and Bo Li. Domain watermark: Effec- tive and harmless dataset copyright protection is closed at hand. Advances in Neural Information Processing Systems , 36, 2023. 1, 2

    2023

  10. [18]

    Zeromark: Towards dataset own- ership verification without disclosing watermark

    Junfeng Guo, Yiming Li, Ruibo Chen, Yihan Wu, Chenxi Liu, and Heng Huang. Zeromark: Towards dataset own- ership verification without disclosing watermark. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 1, 2

    2024

  11. [19]

    news-please: A generic news crawler and extrac- tor

    Felix Hamborg, Norman Meuschke, Corinna Breitinger, and Bela Gipp. news-please: A generic news crawler and extrac- tor. In Proceedings of the 15th International Symposium of Information Science, pages 218–223, 2017. 8, 1

    2017

  12. [20]

    Spectre: Defending against backdoor attacks us- ing robust statistics

    Jonathan Hayase, Weihao Kong, Raghav Somani, and Se- woong Oh. Spectre: Defending against backdoor attacks us- ing robust statistics. InInternational Conference on Machine Learning, pages 4129–4139. PMLR, 2021. 2

    2021

  13. [21]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 3

    2016

  14. [22]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000– 16009, 2022. 2, 3, 5

    2022

  15. [23]

    Introduction to mathematical statistics

    Robert V Hogg, Joseph W McKean, Allen T Craig, et al. Introduction to mathematical statistics . Pearson Education India, 2013. 2, 5

    2013

  16. [24]

    The minipile challenge for data-efficient lan- guage models

    Jean Kaddour. The minipile challenge for data-efficient lan- guage models. arXiv preprint arXiv:2304.08442, 2023. 8, 1

    2023 arXiv

  17. [25]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

    arXiv

  18. [26]

    Learning multiple layers of features from tiny images

    A Krizhevsky. Learning multiple layers of features from tiny images. Master’s thesis, University of Tront, 2009. 1

    2009

  19. [27]

    Albert: A lite bert for self-supervised learning of language representations

    Zhenzhong Lan. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942, 2019. 2, 8

    1909 arXiv

  20. [28]

    An embarrassingly simple backdoor attack on self-supervised learning

    Changjiang Li, Ren Pang, Zhaohan Xi, Tianyu Du, Shoul- ing Ji, Yuan Yao, and Ting Wang. An embarrassingly simple backdoor attack on self-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 4367–4378, 2023. 2, 6

    2023

  21. [29]

    Masked modeling for self-supervised repre- sentation learning on vision and beyond

    Siyuan Li, Luyuan Zhang, Zedong Wang, Di Wu, Lirong Wu, Zicheng Liu, Jun Xia, Cheng Tan, Yang Liu, Baigui Sun, et al. Masked modeling for self-supervised repre- sentation learning on vision and beyond. arXiv preprint arXiv:2401.00897, 2023. 3

    2023 arXiv

  22. [30]

    Untargeted backdoor watermark: Towards harm- less and stealthy dataset copyright protection

    Yiming Li, Yang Bai, Yong Jiang, Yong Yang, Shu-Tao Xia, and Bo Li. Untargeted backdoor watermark: Towards harm- less and stealthy dataset copyright protection. Advances in Neural Information Processing Systems , 35:13238–13250,

  23. [31]

    Black-box dataset ownership verification via backdoor watermarking

    Yiming Li, Mingyan Zhu, Xue Yang, Yong Jiang, Tao Wei, and Shu-Tao Xia. Black-box dataset ownership verification via backdoor watermarking. IEEE Transactions on Informa- tion Forensics and Security, 2023. 1, 2

    2023

  24. [32]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  25. [33]

    Mixmae: Mixed and masked autoencoder for effi- cient pretraining of hierarchical vision transformers

    Jihao Liu, Xin Huang, Jinliang Zheng, Yu Liu, and Hong- sheng Li. Mixmae: Mixed and masked autoencoder for effi- cient pretraining of hierarchical vision transformers. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6252–6261, 2023. 5

    2023

  26. [34]

    Roberta: A robustly optimized bert pretraining approach

    Yinhan Liu. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 364, 2019. 2, 3, 8

    1907 arXiv

  27. [35]

    Wdnet: Watermark- decomposition network for visible watermark removal

    Yang Liu, Zhen Zhu, and Xiang Bai. Wdnet: Watermark- decomposition network for visible watermark removal. In Proceedings of the IEEE/CVF Winter Conference on Appli- cations of Computer Vision, pages 3685–3693, 2021. 2

    2021

  28. [36]

    Stolenencoder: stealing pre-trained encoders in self- supervised learning

    Yupei Liu, Jinyuan Jia, Hongbin Liu, and Neil Zhenqiang Gong. Stolenencoder: stealing pre-trained encoders in self- supervised learning. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Se- curity, pages 2115–2128, 2022. 2

    2022

  29. [37]

    Pixmim: Rethinking pixel reconstruction in masked image modeling

    Yuan Liu, Songyang Zhang, Jiacheng Chen, Kai Chen, and Dahua Lin. Pixmim: Rethinking pixel reconstruction in masked image modeling. arXiv preprint arXiv:2303.02416,

    arXiv

  30. [38]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 3, 5

    2021

  31. [39]

    Dataset inference: Ownership resolution in machine learn- ing

    Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolution in machine learn- ing. arXiv preprint arXiv:2104.10706, 2021. 2

    2021 arXiv

  32. [40]

    Pointer sentinel mixture models

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843, 2016. 5, 8, 1

    2016 arXiv

  33. [41]

    Beit v2: Masked image modeling with vector-quantized visual tokenizers

    Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, and Furu Wei. Beit v2: Masked image modeling with vector-quantized visual tokenizers. arXiv preprint arXiv:2208.06366, 2022. 5

    2022 arXiv

  34. [42]

    Backdoor attacks on self- supervised learning

    Aniruddha Saha, Ajinkya Tejankar, Soroush Abbasi Kooh- payegani, and Hamed Pirsiavash. Backdoor attacks on self- supervised learning. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 13337–13346, 2022. 2

    2022

  35. [43]

    Can’t steal? cont-steal! contrastive stealing attacks against image encoders

    Zeyang Sha, Xinlei He, Ning Yu, Michael Backes, and Yang Zhang. Can’t steal? cont-steal! contrastive stealing attacks against image encoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16373–16383, 2023. 2, 3

    2023

  36. [44]

    Membership inference attacks against machine learning models

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3–18. IEEE, 2017. 3

    2017

  37. [45]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 3

    2014 arXiv

  38. [46]

    Defending against back- door attacks in natural language generation

    Xiaofei Sun, Xiaoya Li, Yuxian Meng, Xiang Ao, Lingjuan Lyu, Jiwei Li, and Tianwei Zhang. Defending against back- door attacks in natural language generation. In Proceed- ings of the AAAI Conference on Artificial Intelligence, pages 5257–5265, 2023. 2

    2023

  39. [47]

    Did you train on my dataset? towards public dataset protection with cleanlabel backdoor watermarking

    Ruixiang Tang, Qizhang Feng, Ninghao Liu, Fan Yang, and Xia Hu. Did you train on my dataset? towards public dataset protection with cleanlabel backdoor watermarking. ACM SIGKDD Explorations Newsletter, 25(1):43–53, 2023. 2

    2023

  40. [48]

    Defending against patch-based backdoor attacks on self- supervised learning

    Ajinkya Tejankar, Maziar Sanjabi, Qifan Wang, Sinong Wang, Hamed Firooz, Hamed Pirsiavash, and Liang Tan. Defending against patch-based backdoor attacks on self- supervised learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages...

    2023

  41. [49]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 6, 1

    2017

  42. [50]

    Masked feature predic- tion for self-supervised visual pre-training

    Chen Wei, Haoqi Fan, Saining Xie, Chao-Yuan Wu, Alan Yuille, and Christoph Feichtenhofer. Masked feature predic- tion for self-supervised visual pre-training. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14668–14678, 2022. 3, 5

    2022

  43. [51]

    Yuechen Xie, Jie Song, Huiqiong Wang, and Mingli Song. Training data provenance verification: Did your model use synthetic data from my generative model for training? In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 23817–23827, 2025. 1

    2025

  44. [52]

    Dataset ownership verification in contrastive pre-trained models,

    Yuechen Xie, Jie Song, Mengqi Xue, Haofei Zhang, Xingen Wang, Bingde Hu, Genlang Chen, and Mingli Song. Dataset ownership verification in contrastive pre-trained models,

  45. [53]

    Simmim: A simple framework for masked image modeling

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9653–9663, 2022. 5

    2022

  46. [54]

    Data poisoning based backdoor attacks to con- trastive learning

    Jinghuai Zhang, Hongbin Liu, Jinyuan Jia, and Neil Zhen- qiang Gong. Data poisoning based backdoor attacks to con- trastive learning. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 24357–24366, 2024. 2

    2024

  47. [55]

    Ssl-cleanse: Tro- jan detection and mitigation in self-supervised learning

    Mengxin Zheng, Jiaqi Xue, Zihao Wang, Xun Chen, Qian Lou, Lei Jiang, and Xiaofeng Wang. Ssl-cleanse: Tro- jan detection and mitigation in self-supervised learning. In European Conference on Computer Vision, pages 405–421. Springer, 2024. 2

    2024

  48. [56]

    Places: A 10 million image database for scene recognition

    Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452–1464, 2017. 5, 1

    2017

  49. [57]

    ibot: Image bert pre-training with online tokenizer

    Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.07832 ,

    arXiv

  50. [58]

    Mimco: Masked image modeling pre-training with con- trastive teacher

    Qiang Zhou, Chaohui Yu, Hao Luo, Zhibin Wang, and Hao Li. Mimco: Masked image modeling pre-training with con- trastive teacher. In Proceedings of the 30th ACM Interna- tional Conference on Multimedia , pages 4487–4495, 2022. 2

    2022

  51. [59]

    A unified membership inference method for visual self-supervised en- coder via part-aware capability

    Jie Zhu, Jirong Zha, Ding Li, and Leye Wang. A unified membership inference method for visual self-supervised en- coder via part-aware capability. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communica- tions Security, pages 1241–1255, 2024. 3, 6 Dataset...

    2024

  52. [60]

    Calculate the t-statistic. First, calculate the mean paired differences between the elements in ∆Rpt and ∆Rvt: d = 1 K XK k=1 (∆Rk pt − ∆Rk vt) (4) where K is the iterations of sampling, ∆Rk pt and ∆Rk vt are the k-th elements in ∆Rpt and ∆Rvt, respectively. Then, calculate th...

  53. [61]

    We first calculate the p-value for the two-tailed test: p = 2P (T >|t|) (7) Then, based on the t-statistic, the final p-value is deter- mined

    Calculate the p-value. We first calculate the p-value for the two-tailed test: p = 2P (T >|t|) (7) Then, based on the t-statistic, the final p-value is deter- mined. When the t-statistic is greater than 0, p = p/2. When the t-statistic is less than 0, p = 1 − p/2

  54. [62]

    Wiki-50k

    Determine significance. Our significance level α is set to 0.05. If p ≤ α, we reject the null hypothesis H0 and consider the suspicious model is illegal. Ifp > α, we fail to reject the null hypothesis and consider the suspicious model is legal. The one-sided pair-wise t-test w...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.