A classifier trained only on transformer fine-tuning data detects an invariant memorization signature that transfers to Mamba, RWKV-4, and RecurrentGemma with AUCs of 0.963, 0.972, and 0.936.
hub
arXiv preprint arXiv:2402.07841 , year=
15 Pith papers cite this work. Polarity classification is still indexing.
15
Pith papers citing it
hub tools
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
The paper introduces a framework for collusion between train- and inference-time adversaries in ML pipelines, proposes a guideline for conjecturing collusion potential, explains prior work, and empirically validates five cases.
MRMMIA is a multi-recall-probe membership inference attack that extracts signals from chat agent memory and outperforms baselines in black-, gray-, and white-box settings.
DistractMIA performs output-only black-box membership inference on vision-language models by inserting semantic distractors and measuring shifts in generated text responses.
Controlled experiments across 96 LoRA adapters show that reduced optimizer updates explain nearly all observed memorization drops in DP-SGD fine-tuning, HMAC pseudonymization cuts exposure 40-61% without creating new targets, and 1-3B models achieve only 0.19-0.28 F1 under the tested budget.
Black-box membership inference on text-to-music models reaches up to 98.6% accuracy by training an auditor on semantic alignment patterns extracted from shadow-model generations.
TRACER presents a semantic-aware framework and the first benchmark for fine-grained code contamination detection across three levels of overlap, reporting F1 scores of 0.91-0.92 and large gains over prior methods.
Distinguishable Deletion unifies knowledge erasure and refusal for LLM unlearning via an energy index that enforces boundaries during training and enables refusal at inference.
Discriminative factorization distinguishes high-quality query sets for black-box model classification, with chance-level error decaying exponentially in query budget and parameters predicting empirical decay rates on auditing tasks.
PopQuiz Attack infers LLM training data membership by turning examples into quiz questions and measuring answer accuracy, reaching 0.873 average ROC-AUC across six models and outperforming prior methods by 20.6%.
CoLA reveals that subset training creates new privacy leakage surfaces via side-channel metadata and model outputs, enabling training-membership and selection-participation membership inference attacks.
RSA prompting enables LLMs to automatically create functional exploits for CVEs in Odoo ERP, succeeding on all tested cases in 3-5 rounds and removing the need for manual effort.
CatShift detects training data membership in LLMs by comparing output shifts induced by fine-tuning on member versus non-member data, relying on catastrophic forgetting without requiring logit access.
Watermark-based dataset inference achieves membership detection performance comparable to loss-based methods when subset exposure is high, under alternate assumptions.
LoRA-MINT uses perplexity to perform membership inference on LoRA-fine-tuned LLMs, reporting 0.77-0.92 precision across four models and three datasets while outperforming baselines.
citing papers explorer
-
SoK: Colluding Adversaries in Machine Learning Pipelines
The paper introduces a framework for collusion between train- and inference-time adversaries in ML pipelines, proposes a guideline for conjecturing collusion potential, explains prior work, and empirically validates five cases.
-
MRMMIA: Membership Inference Attacks on Memory in Chat Agents
MRMMIA is a multi-recall-probe membership inference attack that extracts signals from chat agent memory and outperforms baselines in black-, gray-, and white-box settings.
-
DistractMIA: Black-Box Membership Inference on Vision-Language Models via Semantic Distraction
DistractMIA performs output-only black-box membership inference on vision-language models by inserting semantic distractors and measuring shifts in generated text responses.
-
Decomposing Memorization Reduction in Privacy-Preserving Fine-Tuning of SLMs for CSIRTs
Controlled experiments across 96 LoRA adapters show that reduced optimizer updates explain nearly all observed memorization drops in DP-SGD fine-tuning, HMAC pseudonymization cuts exposure 40-61% without creating new targets, and 1-3B models achieve only 0.19-0.28 F1 under the tested budget.
-
Auditing Training Data in Generative Music Models via Black-Box Membership Inference
Black-box membership inference on text-to-music models reaches up to 98.6% accuracy by training an auditor on semantic alignment patterns extracted from shadow-model generations.
-
TRACER: A Semantic-Aware Framework for Fine-Grained Contamination Detection in Code LLMs
TRACER presents a semantic-aware framework and the first benchmark for fine-grained code contamination detection across three levels of overlap, reporting F1 scores of 0.91-0.92 and large gains over prior methods.
-
Distinguishable Deletion: Unifying Knowledge Erasure and Refusal for Large Language Model Unlearning
Distinguishable Deletion unifies knowledge erasure and refusal for LLM unlearning via an energy index that enforces boundaries during training and enables refusal at inference.
-
Black-box model classification under the discriminative factorization
Discriminative factorization distinguishes high-quality query sets for black-box model classification, with chance-level error decaying exponentially in query budget and parameters predicting empirical decay rates on auditing tasks.
-
Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models
PopQuiz Attack infers LLM training data membership by turning examples into quiz questions and measuring answer accuracy, reaching 0.873 average ROC-AUC across six models and outperforming prior methods by 20.6%.
-
CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Training
CoLA reveals that subset training creates new privacy leakage surfaces via side-channel metadata and model outputs, enabling training-membership and selection-participation membership inference attacks.
-
From Rookie to Expert: Manipulating LLMs for Automated Vulnerability Exploitation in Enterprise Software
RSA prompting enables LLMs to automatically create functional exploits for CVEs in Odoo ERP, succeeding on all tested cases in 3-5 rounds and removing the need for manual effort.
-
Hey, That's My Data! Token-Only Dataset Inference in Large Language Models
CatShift detects training data membership in LLMs by comparing output shifts induced by fine-tuning on member versus non-member data, relying on catastrophic forgetting without requiring logit access.
-
Watermarking for Proprietary Dataset Protection
Watermark-based dataset inference achieves membership detection performance comparable to loss-based methods when subset exposure is high, under alternate assumptions.
-
Auditing Training Data in Domain-adapted LLMs: LoRA-MINT
LoRA-MINT uses perplexity to perform membership inference on LoRA-fine-tuned LLMs, reporting 0.77-0.92 precision across four models and three datasets while outperforming baselines.