Pith. sign in

REVIEW 10 cited by

Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.02936 v4 pith:6VRX2R4D submitted 2024-04-03 cs.CL cs.LG

Jingyang Zhang , Jingwei Sun , Eric Yeats , Yang Ouyang , Martin Kuo , Jianyi Zhang , Hao Frank Yang , Hai Li This is my paper
classification cs.CLcs.LG
keywords datamin-kdetectiondistributionmethodmodelspre-trainingbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The problem of pre-training data detection for large language models (LLMs) has received growing attention due to its implications in critical issues like copyright violation and test data contamination. Despite improved performance, existing methods (including the state-of-the-art, Min-K%) are mostly developed upon simple heuristics and lack solid, reasonable foundations. In this work, we propose a novel and theoretically motivated methodology for pre-training data detection, named Min-K%++. Specifically, we present a key insight that training samples tend to be local maxima of the modeled distribution along each input dimension through maximum likelihood training, which in turn allow us to insightfully translate the problem into identification of local maxima. Then, we design our method accordingly that works under the discrete distribution modeled by LLMs, whose core idea is to determine whether the input forms a mode or has relatively high probability under the conditional categorical distribution. Empirically, the proposed method achieves new SOTA performance across multiple settings. On the WikiMIA benchmark, Min-K%++ outperforms the runner-up by 6.2% to 10.5% in detection AUROC averaged over five models. On the more challenging MIMIR benchmark, it consistently improves upon reference-free methods while performing on par with reference-based method that requires an extra reference model.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Crossing the Margin Cliff: Toward Relearn-Robust LLM Unlearning via Margin Calibration

    cs.AI 2026-07 conditional novelty 7.0 of 10

    Margin Calibration, a non-saturating margin-anchored LoRA polish, crosses the margin cliff and cuts post-attack relearn recovery on all 97 populated cells in the paper's stress matrix.

  2. Averaged Evaluation Masks Capability Trade-Offs: Multi-Source Calibration for High-Sparsity LLM Pruning

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    At 60% SparseGPT sparsity, mixing four capability-matched calibration corpora preserves 58.8% of unpruned performance versus 40.0% for C4 — but the mix is partially calibrated on the same GSM8K and MBPP benchmarks use...

  3. Implicit Reasoning Steering via Concept Chaining

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Reinforcement-learning-optimized concept-chain paragraphs covertly steer language-model multiple-choice preferences after continued pretraining, with far lower detectability than direct paraphrases.

  4. LLM generation novelty through the lens of semantic similarity

    cs.LG 2025-10 conditional novelty 6.0 of 10

    An output is called novel when no semantically similar pretraining context can be retrieved; using this embedding-based measure, SmolLM models show longer-range reuse and instruction tuning increases novelty.

  5. Investigating Training Data Detection in AI Coders

    cs.SE 2025-07 conditional novelty 6.0 of 10

    Most existing training-data detection methods perform poorly on code, while prefix-relative method ReCaLL consistently scores highest, though all degrade under code mutations.

  6. Identifying Pre-training Data in LLMs: A Neuron Activation-Based Detection Framework

    cs.AI 2025-07 reject novelty 6.0 of 10

    NA-PDD detects pre-training data in LLMs by comparing which neurons activate for a test text against neurons linked to known training versus non-training texts, and claims large AUC improvements on three benchmarks.

  7. MrM: Black-Box Membership Inference Attacks against Multimodal RAG Systems

    cs.CV 2025-06 conditional novelty 6.0 of 10

    MrM is a black-box membership inference attack on multimodal RAG systems that masks key objects in a target image and uses the system's ability to reconstruct them as a membership signal.

  8. Membership Inference Attacks on Tokenizers of Large Language Models

    cs.CR 2025-10 conditional novelty 5.0 of 10

    Tokenizers leak dataset membership: using distinctive tokens that appear in a dataset's text, an attacker can detect with AUC up to 0.77 whether that dataset was part of tokenizer training.

  9. Neural Breadcrumbs: Membership Inference Attacks on LLMs Through Hidden State and Attention Pattern Analysis

    cs.LG 2025-09 conditional novelty 5.0 of 10

    A random forest trained on transformer hidden-state and attention features detects training data membership with about 0.83 average AUC, far above output-based attacks.

  10. SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks

    cs.CR 2025-06 conditional novelty 5.0 of 10

    SOFT paraphrases low-loss fine-tuning samples before training, reducing MIA AUC from about 0.82 to about 0.54 across six datasets at roughly 7% perplexity cost.

Pith tools