Pith. sign in

REVIEW 4 major objections 5 minor 47 references

MrM: Black-Box Membership Inference Attacks against Multimodal RAG Systems

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that an attacker with only image inputs and text outputs can determine whether a target image is stored in a multimodal RAG knowledge base, with sample-level AUC above 80 percent on most of eight commercial…

desk verdict A genuinely new attack setting with broad experiments, but the method write-up leaves key parameters (pt, Q, evaluator weights) unspecified and the geometric model ignores truncation; the label-access worry in the stress test is a misread. read the letter →

arxiv 2506.07399 v1 pith:QC2D3VFE submitted 2025-06-09 cs.CV cs.AI

Peiru Yang , Jinhua Yin , Haoran Zheng , Xueying Bai , Huili Wang , Yufei Sun , Xintian Li , Shangguang Wang
show 2 more authors
Yongfeng Huang Tao Qi
This is my paper
classification cs.CVcs.AI
keywords membershipinferenceattackmultimodalretrieval-augmentedgenerationvision-languagemodelsblack-boxobject-awareperturbationmaskselectiongeometrichypothesistestingprivacyleakage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MrM is presented as the first black-box membership inference framework aimed at multimodal retrieval-augmented generation (RAG) systems. The paper's central claim is that an attacker who can only submit images and read text responses can decide whether a target image is stored in the underlying knowledge base, by masking individual detected objects, asking the system to name the hidden content, and counting how many tries the system needs before it succeeds. Because retrieval supplies the answer for stored images but not for others, the trial counts separate members from non-members. On eight commercial vision-language models and two datasets, the reported sample-level AUC (area under the ROC curve) reaches 80–85 percent on most models, and set-level AUC approaches 1.0 once ten images are aggregated. If the claim holds, private image databases behind deployed multimodal assistants are exposed to membership probing through the public API.

What carries the argument

The argument rides on a three-part mechanism. First, object-aware perturbation masks detected objects instead of arbitrary patches, because objects are small enough not to block retrieval yet carry independent semantics that can be verbalized. Second, counterfact-informed mask selection computes, with a proxy vision-language model, a feature vector per mask containing target confidence $p_c$, confidence gap $\Delta p = \max(P)-p_c$, entropy $H$, and top-$k$ probabilities; masks with low target confidence, small confidence gap, and high entropy are ranked as informative, meaning the target model cannot answer them from memory. Third, statistical inference models each mask's query count as geometric: $x_j \sim \mathrm{Geometric}(p_t)$ under membership, the aggregate $S = \sum_{j=1}^{K} x_j$ has mean $K/p_t$ and variance $K(1-p_t)/p_t^2$, and the test statistic $p = 1 - \Phi\left((S - K/p_t)/\sqrt{K(1-p_t)/p_t^2}\right)$ decides whether to reject the null hypothesis. The central identity is the geometric-trial model, because it converts otherwise noisy textual answers into a single number with a known null distribution.

What would settle it

Rerun MrM on the same eight commercial models with the mask selector restricted to proxy-predicted labels rather than ground-truth categories, while keeping everything else fixed. If the variant's AUC stays near the reported level of about 80 percent, the attack stands under the stated threat model; if it drops substantially (for example toward the ablation without the proxy selector), the headline results rely on label access the black-box attacker is not granted.

Watch

Extended reading notes

Core claim

The paper works from a simple asymmetry: when a target image is in the RAG database, the retriever can hand the generator the missing semantic content, so the system names a masked object quickly and correctly; when the image is absent, the generator must rely on its own learned knowledge and is more likely to hesitate or hallucinate. MrM turns this asymmetry into a statistical test. It detects salient objects, masks them one at a time to keep the rest of the image retrievable yet strip the answer from the input, scores each mask with a counterfactual proxy vision-language model to select regions the target model cannot fill from self-knowledge, and repeatedly queries until the first correct reconstruction. Under the null hypothesis that the image is a member, the number of trials for each mask follows a geometric distribution, so the aggregate trial count is approximately normal and yields a p-value. The paper reports that this procedure attains AUC above 80 percent for most of eight commercial systems, remains effective under a cautionary system prompt, and degrades only modestly when the database images have been flipped, gray-scaled, cropped, or blurred.

Load-bearing premise

The attack's load-bearing premise is that the attacker knows the true category of each masked region, because the counterfact-informed selector scores masks with the ground-truth target confidence $p_c$; the stated threat model gives the attacker no such labels, and the appendix states the COCO ground-truth annotations are not used.

Editorial extensions

If this is right

  • Text-only RAG membership attacks that ask directly or compare similarity are substantially weaker, so the attack's power comes from the masked-object design rather than from model compliance.
  • A defender cannot rely on a simple system prompt saying not to reveal membership, since MrM still scores above 80 percent AUC on most of the eight tested systems.
  • Image-level transformations such as flipping, grayscaling, cropping, and blurring do not neutralize the attack; the augmentation-aware variant keeps high discrimination.
  • Attacks improve with set size: aggregating ten images is usually enough to reach an AUC close to 1.0, meaning an adversary can identify a batch of leaked images with near certainty.
  • Smaller open-source vision-language models show weaker but still non-trivial signals, suggesting the attack generalizes across model scales while depending on reasoning capability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the reported method depends on knowing the true category of each masked object to compute the target confidence; an attacker in the stated threat model would need to estimate that category from the proxy model itself, and the paper does not provide that variant.
  • Editorial extension: the geometric-trial formulation is modality-agnostic, so the same first-correct-trial statistic could be applied to audio or video RAG systems once detect-and-mask units exist for those modalities.
  • Editorial extension: the robustness results suggest a concrete defensive direction the paper does not test: equalizing trial counts by injecting retrieval noise or refusing to ground masked-region questions would suppress the signal directly.
  • Editorial extension: object detection defines the attack's scope; in domains without clean object semantics, such as medical scans or satellite imagery (which the paper's limitation section names), both mask selection and self-knowledge suppression would need redesign.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MrM, claimed to be the first black-box membership inference attack framework for multimodal retrieval-augmented generation (RAG) systems. The method masks detected objects in a target image, uses a proxy VLM to select 'informative' masks via a counterfactual informativeness score, and then queries the black-box RAG system with the perturbed images. Membership is decided by a hypothesis test: the number of trials until the system correctly identifies a masked object is modeled as geometric, and the sum over masks is approximated as normal via the CLT to produce a p-value. Experiments on COCO and Flickr with eight commercial LVLMs report strong sample-level AUC (mostly above 80%) and near-perfect set-level AUC at K=10, plus robustness to image transformations and a cautionary system prompt.

Significance. If the claims hold, the paper is significant: it would demonstrate that textual outputs of a black-box multimodal RAG system can leak membership information about images in the retrieval database, a timely and practically relevant privacy risk. The empirical scope is a strength: two datasets, eight commercial models, set-level aggregation, adaptive defenses, and a supplementary study on open-source small models. The paper also ships useful qualitative case studies and explicitly discusses limitations in domain generalization. However, several load-bearing details of the attack are underspecified, especially the source of object labels for mask scoring and trial scoring, the value or estimation of the null success probability p_t, and the mismatch between the capped query protocol and the unbounded geometric model. These issues must be resolved before the black-box claim is fully convincing.

major comments (4)
  1. [Section 4.3 and Appendix B.3] The mask-selection procedure requires the proxy VLM's probability for 'the ground-truth category of the masked region' (pc), and Section 4.4 requires knowing whether each response 'correctly identifies the masked object'. The paper never states how the attacker obtains the category label for each mask. SAM2 produces masks but not semantic category labels, and Appendix B.3 explicitly says COCO ground-truth annotations are not used. Since the attacker possesses the target image, a possible source is the attacker's own visual knowledge or an auxiliary classifier, but this must be stated and validated. If the experiments instead used dataset annotations for pc and for correctness scoring, the reported results would not demonstrate a black-box attack.
  2. [Section 4.4, Eq. (2)] The p-value in Eq. (2) depends on p_t, the per-mask success probability under H0, but the paper specifies no value, no estimation procedure, and no attacker-available calibration method for p_t. As written, the decision rule is undefined because p_t appears directly in the standardization of S. The authors should specify how p_t is obtained (for example, from non-member probe queries or a held-out calibration set) or reformulate the test to avoid dependence on an unspecified parameter.
  3. [Section 4.4 and Figure 1] Figure 1 states that the attacker queries 'at most Q times' per mask, but Section 4.4 defines x_j as the number of trials until the first correct prediction and models it as an unbounded Geometric(p_t) random variable. With a query cap, x_j follows a truncated geometric distribution, and the additive CLT used to approximate S in Eq. (2) does not apply. This mismatch can materially miscalibrate the p-value. Please either remove the cap from the protocol or provide a truncated/distributional analysis of S.
  4. [Section 4.3] The 'rule-based evaluator' is described only qualitatively: masks are ranked by an ensemble of normalized feature scores favoring high entropy, low target confidence, and small confidence gap. The exact scoring function, feature weights, normalization scheme, the value of k in the top-k distribution feature, the number of masks selected, and the query cap Q are not specified. These choices directly influence which masks are queried and therefore the reported AUC. Without these definitions the method is not reproducible; please provide the full selection rule and an ablation over its hyperparameters.
minor comments (5)
  1. [Table 1] The table header contains a typo: 'Fickr' should be 'Flickr'.
  2. [Section 4.2 and Section 5.2] The symbol K is used both for the number of detected objects/masks (Section 4.2, 4.4) and for the set size in the set-level ROC evaluation (Section 5.2, Figure 2). Please use distinct notation, e.g., m for masks and K for set size.
  3. [Section 5.3 and Figure 3] The ablation results are presented only as radar charts without numeric AUC or TPR@5%FPR values. Please report the corresponding numbers and standard deviations, since the overlapping radar axes make it difficult to verify the claimed performance drops.
  4. [Section 5.2 and Figure 2] The ROC curves are described as based on 200 random samples per K and repeated five times, but no confidence intervals or error bars are shown. Given that commercial API responses can be stochastic, please report variance or confidence bands for the AUC values.
  5. [Appendix C, Figure 6] The open-source small-model results are described as showing a 'modest drop' relative to commercial models, but no quantitative comparison to the commercial results is provided in the same figure or table. Adding numeric values and the corresponding commercial-model scores would make the comparison concrete.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the attack's empirical evaluation is not definitionally tied to its inputs; identified gaps are under-specification and label-availability issues, not self-referential derivations.

full rationale

MrM is an empirical attack pipeline whose claimed prediction is a membership decision built from query-response statistics. The mask-informativeness score and trial-scoring rule do use the ground-truth object category of a masked region (Sections 4.3 and 4.4), and the null distribution in Eq. (2) depends on an unspecified member success probability pt. These are serious reproducibility and threat-model gaps, because the attacker is denied access to such labels and no estimation procedure for pt is given, and Appendix B.3 explicitly says COCO ground-truth annotations are not used. However, these are not circularity in the sense required by the review rules: the derived membership decision does not reduce by construction to the inputs of the mask-selection rule, and no fitted parameter is renamed as a prediction. The mask-selection heuristic is defined from proxy-model confidence and entropy rather than tuned to membership labels, and the final inference compares response trial counts to a geometric null distribution rather than to the outcome being predicted. The paper also does not rely on a load-bearing self-citation chain or an imported uniqueness theorem; the attack is benchmarked against external baselines and commercial APIs. Under the rule that under-specification and unavailable-input problems are correctness risks rather than circularity, the appropriate finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities, only a procedure (counterfactual mask selection). The main unstated inputs are the ground-truth object categories and the null success probability pt, both of which are load-bearing but not sourced from the threat model.

free parameters (5)
  • p_t (null success probability) = unspecified
    Used in Eq. (1)-(2) to define the null distribution of S. The paper never states how an attacker would set p_t; in practice it must be estimated, probably from the same test data, introducing a fitted parameter.
  • Q (max queries per mask) = unspecified
    Figure 1 and the query procedure say 'Query for at most Q times', truncating the geometric distribution. The value of Q is not given and changes the statistic's distribution.
  • Rule-based evaluator weights/thresholds = unspecified
    Section 4.3 ranks masks using an ensemble of normalized feature scores with no exact weights or thresholds, likely tuned by the authors on the evaluation data.
  • Top-k (distribution feature) = unspecified
    The feature vector includes top-k probabilities {p(i)} with k unspecified.
  • Number of selected masks = unspecified
    The paper does not state how many of the highest-ranked masks are used per image or how masks are aggregated.
assumptions (5)
  • domain assumption Independent, unbounded geometric trials for each mask
    Section 4.4 models x_j as Geometric(pt). This requires trials to be independent and unlimited, but the procedure caps queries at Q and model responses may be affected by prior queries.
  • domain assumption Ground-truth object categories are available for masked regions
    Section 4.3 requires pc, the probability of the ground-truth category, to compute confidence gap. No source for these labels is given; Appendix B.3 says COCO annotations are not used.
  • ad hoc to paper Proxy VLM uncertainty indicates target VLM self-knowledge
    The counterfactual mask selection assumes that masks which confuse the 7B Qwen proxy will also suppress reconstruction from the target VLM's memorized knowledge; this is a conjecture, not demonstrated.
  • domain assumption Retrieval succeeds for partially masked members
    The core intuition is that masking small objects leaves enough visual signal for CLIP retrieval to find the original image when it is a member. This is plausible but not guaranteed, especially under the transformations tested in Section 5.4.
  • standard math CLT approximation for aggregated trials
    Eq. (2) invokes the central limit theorem for S, which is standard but requires large K and finite variance; K (number of masks) is small in practice (likely under 10).

how reviews work

0 comments
Cite this review

Pith. "Pith review of MrM: Black-Box Membership Inference Attacks against Multimodal RAG Systems." pith.science (2026). https://pith.science/paper/QC2D3VFE

@misc{pith2026250607399,
  author       = {Pith},
  title        = {Pith review of: MrM: Black-Box Membership Inference Attacks against Multimodal RAG Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QC2D3VFE}},
  note         = {Machine review of arXiv:2506.07399}
}
read the original abstract

Multimodal retrieval-augmented generation (RAG) systems enhance large vision-language models by integrating cross-modal knowledge, enabling their increasing adoption across real-world multimodal tasks. These knowledge databases may contain sensitive information that requires privacy protection. However, multimodal RAG systems inherently grant external users indirect access to such data, making them potentially vulnerable to privacy attacks, particularly membership inference attacks (MIAs). % Existing MIA methods targeting RAG systems predominantly focus on the textual modality, while the visual modality remains relatively underexplored. To bridge this gap, we propose MrM, the first black-box MIA framework targeted at multimodal RAG systems. It utilizes a multi-object data perturbation framework constrained by counterfactual attacks, which can concurrently induce the RAG systems to retrieve the target data and generate information that leaks the membership information. Our method first employs an object-aware data perturbation method to constrain the perturbation to key semantics and ensure successful retrieval. Building on this, we design a counterfact-informed mask selection strategy to prioritize the most informative masked regions, aiming to eliminate the interference of model self-knowledge and amplify attack efficacy. Finally, we perform statistical membership inference by modeling query trials to extract features that reflect the reconstruction of masked semantics from response patterns. Experiments on two visual datasets and eight mainstream commercial visual-language models (e.g., GPT-4o, Gemini-2) demonstrate that MrM achieves consistently strong performance across both sample-level and set-level evaluations, and remains robust under adaptive defenses.

Figures

Figures reproduced from arXiv: 2506.07399 by the authors.

Figure 1
Figure 1. The overall framework of the proposed MrM method. It first perturbs the target image via [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. ROC curves for set-level MIAs with varying set sizes ( [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Ablation study results visualized as radar charts. We compare the full MrM method with [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Robustness of MrM against adaptive image-level transformations applied to the database, [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Illustration of the black-box MIA against a multimodal RAG system. An attacker queries [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Membership inference performance of MrM on locally deployed open-source vision [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Set-level ROC curves comparing MrM with its object-agnostic variant under different set [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Case study demonstrating the advantage of MrM in distinguishing between in-database [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 28 canonical work pages

  1. [1]

    Murag: Multimodal retrieval-augmented generator for open question answering over images and text

    Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William Cohen. Murag: Multimodal retrieval-augmented generator for open question answering over images and text. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 5558– 5570, 2022

    work page 2022

  2. [2]

    Universal vision- language dense retrieval: Learning a unified representation space for multi-modal retrieval

    Zhenghao Liu, Chenyan Xiong, Yuanhuiyi Lv, Zhiyuan Liu, and Ge Yu. Universal vision- language dense retrieval: Learning a unified representation space for multi-modal retrieval. In The Eleventh International Conference on Learning Representations , 2023

    work page 2023

  3. [3]

    Retrieval-augmented multimodal language modeling

    Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Retrieval-augmented multimodal language modeling. In Proceedings of the 40th International Conference on Machine Learning , pages 39755–39769, 2023

    work page 2023

  4. [4]

    A survey of vision-language pre-trained models

    Yifan Du, Zikang Liu, Junyi Li, and Wayne Xin Zhao. A survey of vision-language pre-trained models. arXiv preprint arXiv:2202.10936, 2022

    arXiv 2022

  5. [5]

    Vision-language models for vision tasks: A survey

    Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    work page 2024

  6. [6]

    In-context learning enables multimodal large language models to classify cancer pathology images

    Dyke Ferber, Georg Wölflein, Isabella C Wiest, Marta Ligero, Srividhya Sainath, Narmin Ghaffari Laleh, Omar SM El Nahhas, Gustav Müller-Franzes, Dirk Jäger, Daniel Truhn, et al. In-context learning enables multimodal large language models to classify cancer pathology images. Nature Communications, 15(1):10104, 2024

    work page 2024

  7. [7]

    Mmed-rag: Versatile multimodal rag system for medical vision language models

    Peng Xia, Kangyu Zhu, Haoran Li, Tianze Wang, Weijia Shi, Sheng Wang, Linjun Zhang, James Zou, and Huaxiu Yao. Mmed-rag: Versatile multimodal rag system for medical vision language models. In Neurips Safe Generative AI Workshop, 2024

    work page 2024

  8. [8]

    Rule: Reliable multimodal rag for factuality in medical vision language models

    Peng Xia, Kangyu Zhu, Haoran Li, Hongtu Zhu, Yun Li, Gang Li, Linjun Zhang, and Huaxiu Yao. Rule: Reliable multimodal rag for factuality in medical vision language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages 1081–1093, 2024

    work page 2024

Show all 47 references
  1. [9]

    Retrieval-augmented generation for knowledge-intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems...

    2020

  2. [10]

    Towards trustworthy retrieval augmented generation for large language models: A survey

    Bo Ni, Zheyuan Liu, Leyao Wang, Yongjia Lei, Yuying Zhao, Xueqi Cheng, Qingkai Zeng, Luna Dong, Yinglong Xia, Krishnaram Kenthapadi, et al. Towards trustworthy retrieval augmented generation for large language models: A survey. arXiv preprint arXiv:2502.06872, 2025

    2025 arXiv

  3. [11]

    The good and the bad: Exploring privacy issues in retrieval- augmented generation (rag)

    Shenglai Zeng, Jiankun Zhang, Pengfei He, Yiding Liu, Yue Xing, Han Xu, Jie Ren, Yi Chang, Shuaiqiang Wang, Dawei Yin, et al. The good and the bad: Exploring privacy issues in retrieval- augmented generation (rag). In Findings of the Association for Computational Linguistics A...

    2024

  4. [12]

    Membership inference attacks against machine learning models

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3–18. IEEE, 2017

    2017

  5. [13]

    Demystifying mem- bership inference attacks in machine learning as a service

    Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei. Demystifying mem- bership inference attacks in machine learning as a service. IEEE transactions on services computing, 14(6):2073–2089, 2019

    2019

  6. [14]

    Membership inference attacks on machine learning: A survey.ACM Computing Surveys (CSUR), 54(11s):1–37, 2022

    Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. Membership inference attacks on machine learning: A survey.ACM Computing Surveys (CSUR), 54(11s):1–37, 2022. 10

    2022

  7. [15]

    Membership inference attacks from first principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In 2022 IEEE symposium on security and privacy (SP), pages 1897–1914. IEEE, 2022

    2022

  8. [16]

    Label- only membership inference attacks

    Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label- only membership inference attacks. In International conference on machine learning , pages 1964–1974. PMLR, 2021

    1964

  9. [17]

    Membership inference attack on graph neural networks

    Iyiola E Olatunji, Wolfgang Nejdl, and Megha Khosla. Membership inference attack on graph neural networks. In 2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) , pages 11–20. IEEE, 2021

    2021

  10. [18]

    Is my data in your retrieval database? membership inference attacks against retrieval augmented generation

    Maya Anderson, Guy Amit, and Abigail Goldsteen. Is my data in your retrieval database? membership inference attacks against retrieval augmented generation. arXiv preprint arXiv:2405.20446, 2024

    2024 arXiv

  11. [19]

    Generating is believing: Membership inference attacks against retrieval-augmented generation

    Yuying Li, Gaoyang Liu, Chen Wang, and Yang Yang. Generating is believing: Membership inference attacks against retrieval-augmented generation. In ICASSP 2025-2025 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 1–5. IEEE, 2025

    2025

  12. [20]

    Mask-based membership inference attacks for retrieval-augmented generation

    Mingrui Liu, Sixiao Zhang, and Cheng Long. Mask-based membership inference attacks for retrieval-augmented generation. In Proceedings of the ACM on Web Conference 2025 , pages 2894–2907, 2025

    2025

  13. [21]

    Riddle me this! stealthy membership inference for retrieval-augmented generation

    Ali Naseh, Yuefeng Peng, Anshuman Suri, Harsh Chaudhari, Alina Oprea, and Amir Houmansadr. Riddle me this! stealthy membership inference for retrieval-augmented generation. arXiv preprint arXiv:2502.00306, 2025

    2025 arXiv

  14. [22]

    Transformers in vision: A survey

    Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah. Transformers in vision: A survey. ACM computing surveys (CSUR), 54 (10s):1–41, 2022

    2022

  15. [23]

    An empirical study of training end-to-end vision-and-language transformers

    Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang, Lijuan Wang, Chenguang Zhu, Pengchuan Zhang, Lu Yuan, Nanyun Peng, et al. An empirical study of training end-to-end vision-and-language transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and ...

    2022

  16. [24]

    Vision+ language applications: A survey

    Yutong Zhou and Nobutaka Shimada. Vision+ language applications: A survey. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 826–842, 2023

    2023

  17. [25]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision , pages 4015–4026, 2023

    2023

  18. [26]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , p...

    2021

  19. [27]

    Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks

    Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems, 32, 2019

    2019

  20. [28]

    Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pages 12888–12900. PMLR, 2022

    2022

  21. [29]

    Colpali: Efficient document retrieval with vision language models

    Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. Colpali: Efficient document retrieval with vision language models. In The Thirteenth International Conference on Learning Representations , 2024. 11

    2024

  22. [30]

    M 4I: Multi-modal models membership inference

    Pingyi Hu, Zihan Wang, Ruoxi Sun, Hu Wang, and Minhui Xue. M 4I: Multi-modal models membership inference. Advances in Neural Information Processing Systems , 35:1867–1882, 2022

    2022

  23. [31]

    Practical membership inference attacks against large-scale multi-modal models: A pilot study

    Myeongseob Ko, Ming Jin, Chenguang Wang, and Ruoxi Jia. Practical membership inference attacks against large-scale multi-modal models: A pilot study. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4871–4881, 2023

    2023

  24. [32]

    Membership inference attacks against large vision-language models

    Zhan Li, Yongtao Wu, Yihang Chen, Francesco Tonin, Elias Abad Rocamora, and V olkan Cevher. Membership inference attacks against large vision-language models. Advances in Neural Information Processing Systems, 37:98645–98674, 2024

    2024

  25. [33]

    Lumia: Linear probing for unimodal and multimodal membership inference attacks leveraging internal llm states

    Luis Ibanez-Lissen, Lorena Gonzalez-Manzano, Jose Maria de Fuentes, Nicolas Anciaux, and Joaquin Garcia-Alfaro. Lumia: Linear probing for unimodal and multimodal membership inference attacks leveraging internal llm states. arXiv preprint arXiv:2411.19876, 2024

    2024 arXiv

  26. [34]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings...

    2014

  27. [35]

    From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the association for computational linguistics , 2:67–78, 2014

    2014

  28. [36]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  29. [37]

    Gemini 2.0: Flash, flash-lite and pro, February 2025

    Shrestha Basu Mallick and Logan Kilpatrick. Gemini 2.0: Flash, flash-lite and pro, February 2025. URL https://developers.googleblog.com/zh-hans/ gemini-2-family-expands/ . Accessed: 2025-05-01

    2025

  30. [38]

    Introducing claude 3.5 sonnet, June 2024

    Anthropic. Introducing claude 3.5 sonnet, June 2024. URL https://www.anthropic.com/ news/claude-3-5-sonnet . Accessed: 2025-05-01

    2024

  31. [39]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools, 2024

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie H...

    2024

  32. [40]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  33. [41]

    Pixtral 12b

    Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, et al. Pixtral 12b. arXiv preprint arXiv:2410.07073, 2024

    arXiv

  34. [42]

    Kimi chat, 2024

    Moonshot AI. Kimi chat, 2024. URL https://kimi.moonshot.cn/. Accessed: 2025-04-10

    2024

  35. [43]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024

    2024 arXiv

  36. [44]

    Detecting pretraining data from large language models

    Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations. 12

  37. [45]

    Min-k%++: Improved baseline for detecting pre-training data from large language models

    Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Frank Yang, and Hai Li. Min-k%++: Improved baseline for detecting pre-training data from large language models. arXiv preprint arXiv:2404.02936, 2024

    2024 arXiv

  38. [46]

    You only look once: Unified, real-time object detection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016

    2016

  39. [47]

    The image shows\

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre- Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library. 2024. 13 A Detailed Threat Model Target System Response RAG Membership Inference Attack (MIA) Suspected Ima...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.