{"id":"52927d46-90fa-49b0-96ac-8b98d945107e","arxiv_id":"2506.19399","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"VeilProbe claims automatic black-box detection of LLM pre-training text, but its reported gains likely stem from a transductive evaluation where the feature extractor is trained on the test texts.","lead":"VeilProbe is a proposed framework that detects whether a text was part of a black-box LLM's training data by learning to imitate the LLM's continuations and classifying internal features of the imitation. The paper reports very high accuracy, but the evaluation appears to train the imitation model on the same texts it later classifies, so the results are not trustworthy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation protocol trains the seq2seq feature extractor on the same texts it later scores, so reported AUCs may reflect transductive memorization rather than a generalizable black-box detector.","rationale":"The reader identifies the same weak spot I would flag first: the absence of any split between the texts used to train the sequence-to-sequence mapping model and the texts later scored. I agree that this is the load-bearing assumption behind the reported AUCs and TPR values. It is not a claim that the authors are hiding information; the paper simply does not describe the required evaluation protocol, and the text as written implies that all candidate texts are used to train the feature extractor. This makes the evaluation transductive: the classifier can rely on the mapping model having already fit each test pair. That does not by itself invalidate a transductive protocol, but the paper does not present the method as limited to a fixed, pre-specified candidate pool, nor does it disclose this protocol as a limitation. A clean held-out test would settle whether the features generalize. The released code is a useful asset, but it does not remedy the missing split in the paper. Secondary issues such as the p-value threshold selection and missing error bars are less fundamental and would not alone justify rejection. My read therefore leaves the reader's verdict unchanged.","tokens_in":18898,"tokens_out":6886,"duration_ms":85230,"concrete_test":"Fix one setting, e.g., WikiMIA with Pythia-6.9B. Split the candidate set S before any training. Train the seq2seq mapping model only on the training split's ⟨s,sout⟩ and ⟨r,rout⟩ pairs; freeze it; extract features for the held-out split. Train the prototype classifier on the same 50 ground-truth samples (drawn from the training split) and evaluate AUC and TPR@5%FPR only on held-out texts. Compare with the paper's all-pairs protocol under identical hyperparameters. If the held-out AUC drops materially (say more than 0.05-0.1), the reported gains are transductive artifacts; if it holds, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption behind the headline numbers is that the membership features generalize to texts the mapping model has not already fit. The paper never establishes this. Section 4.1 builds ⟨s,sout⟩ and ⟨r,rout⟩ pairs for 'each text s∈S'; Section 4.2 says the mapping model 'is trained on ⟨s,sout⟩ pairs' and hidden states are then extracted for those same inputs; Section 5.1 reserves only about 50 ground-truth samples for the prototype classifier, with the rest used as 'texts to be detected.' No split between mapping-model training and evaluation is described. Thus every test feature Z=Zs⊕Zec comes from a surrogate that has seen the exact s→sout mapping of the test text. The classifier then only has to exploit pair-specific artifacts of the mapping model, not a generalizable text-only membership signal. The claim to detect 'a given text' in a black-box setting is also not supported: a newly arriving text would first have to be added to the mapping-model training pool before it can be scored. This is the weakest link in the central argument; the high AUCs are otherwise difficult to reconcile with the much weaker grey-box baselines that use exact token probabilities.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VeilProbe, a framework for detecting whether a given text was part of a target LLM's pre-training data in a black-box setting where only input and output texts are accessible. The method first samples text-to-suffix pairs (including perturbed texts) from the target LLM, then trains a sequence-to-sequence mapping model to extract membership features from the hidden states of those pairs, combines them with perturbation calibration features, and finally uses a prototype-based classifier with an Information Bottleneck objective to classify texts. The authors report strong results on WikiMIA, BookTection, and arXivTection, outperforming grey-box and black-box baselines on both sentence-level and document-level detection.","tokens_in":19111,"tokens_out":4553,"duration_ms":47810,"significance":"The problem is practically important: detecting pre-training data in closed models is relevant to privacy, copyright, and data-contamination auditing. A fully automatic text-only detector would be a meaningful advance over existing black-box methods that require handcrafted instructions, and it could provide a complement to grey-box approaches that need token probabilities. The proposed design is thoughtful in combining a learned text-to-suffix mapping with perturbation calibration and a few-shot prototype classifier. If the evaluation is properly validated with a non-transductive protocol, the contribution would likely be significant for the community. The code is made available, which supports reproducibility of the re-run experiments.","major_comments":[{"comment":"The evaluation appears to be transductive: the sequence-to-sequence mapping model is trained on the text-to-suffix pairs for 'each text s∈S' (Section 4.1) and then hidden states are extracted for the same texts (Section 4.2), while Section 5.1 only reserves approximately 50 ground-truth samples for the prototype classifier and treats the remaining samples as the texts to be detected. No disjoint split between mapping-model training and feature extraction is described. Consequently, the features used to evaluate a test text come from a surrogate model that has already fitted that exact text's continuation pattern, so the reported AUCs (Tables 2 and 3) may reflect pair-specific memorization rather than a generalizable text-only membership signal. Please re-run the experiments with a clean split (for example, train the mapping model only on the 50 ground-truth samples or on a separate training subset, then extract features for held-out texts), or provide evidence that the features for held-out texts are not materially degraded relative to the transductive ones.","section":"§4.2, §5.1"},{"comment":"The significance-test-based critical feature selection uses the ground-truth labels G1 and G0, but the paper does not state whether this test is restricted to the approximately 50 training samples or applied to the entire candidate pool. If the p-value threshold is chosen using features from the test texts, the labels of those test texts influence the final feature representation, which would be a form of label leakage. Please clarify that the significance test is performed only on the labeled training subset, and if it is currently applied more broadly, rerun the experiments accordingly.","section":"§4.2, Eqs. (3)–(4)"},{"comment":"Because the mapping model is trained on text-to-suffix pairs for all candidate texts, including the test texts, the comparison against grey-box baselines is not apples-to-apples: VeilProbe effectively uses unlabeled input-output pairs from the test pool to train its feature extractor, whereas the baselines only use the token-level information of the test text itself. If a proper split is introduced, the claimed 'good margin' over grey-box methods may shrink. The manuscript should either report the held-out evaluation explicitly or discuss how the transductive training of the feature extractor affects the comparison.","section":"§5.2, Tables 2–3"}],"minor_comments":[{"comment":"The caption contains a typo: 'comparation' should be 'comparison'.","section":"Figure 1 caption"},{"comment":"The baseline name 'FeartureAgg' should be 'FeatureAgg'.","section":"Table 2"},{"comment":"The value of β in the Information Bottleneck objective is not reported or ablated in the experiments; please specify how β is set and whether the results are sensitive to it.","section":"§4.3, Eq. (8)"},{"comment":"Section 5.1 says 'three suffixes' were generated per text, while Section 4.1 says 'more than one suffix' without specifying the number; the text should be consistent.","section":"§5.1 and §4.1"},{"comment":"The text says 'GLMs' where it should say 'LLMs'.","section":"Appendix C.2"}],"recommendation":"major_revision","confidential_remarks":"The transductive evaluation is the key concern. The reader's report and my own reading of Sections 4.1, 4.2, and 5.1 strongly suggest that the mapping model is trained on the very texts it later scores. This is a load-bearing issue because the paper's central claim is that VeilProbe 'automatically detects' pre-training texts in a black-box setting, which requires generalization to texts not used in feature-extractor training. If the authors can provide a clean held-out evaluation (e.g., a separate mapping-model training split) and the results hold, the paper could be a strong contribution. If the results collapse under such a split, the paper's main empirical claims would be unsupported. The p-value feature selection is another point that needs clarification. I therefore recommend major revision, asking for a proper evaluation protocol before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Xiang — quick take on arXiv:2506.19399. The framework is clever, but the empirical claims don't hold up as submitted. The surrogate seq2seq feature extractor appears to be trained on the same ⟨s,sout⟩ pairs that are later scored, so the AUCs around 0.91–0.96 may just reflect transductive memorization, not a generalizable black-box detector.\n\nWhat's genuinely new is the assembly: surrogate seq2seq to model the text-to-suffix mapping, key-token perturbation with proxy-model attributions, and a prototype classifier with information bottleneck. That's a reasonable pipeline for black-box MIA, and the paper is upfront about using only text outputs. The ablations are thorough, and the document-level results are consistent with the sentence-level pattern. The writing is clear.\n\nThe load-bearing problem is the missing split. Section 4.1 builds pairs for 'each text s∈S'; Section 4.2 trains the mapping model on those pairs; Section 5.1 only reserves ~50 ground-truth samples for the prototype classifier. Nothing in the paper indicates that the mapping model is trained on a disjoint set. If the feature extractor has already fit each test text's continuation pattern, the classifier only needs to exploit artifacts of that fit. That would explain why VeilProbe beats grey-box methods that use exact token probabilities—a result that is otherwise hard to believe. The p-value threshold is selected from a small grid on the test data, with no validation split; error bars are absent; and the 'first automatic black-box detector' claim is undercut by Oren et al. 2023, which they cite. These are fixable if the authors retrain the mapping model on a held-out corpus and report the split.\n\nThe central idea is worth a serious look, but the evidence as presented doesn't support the central claim. The paper deserves peer review—a good referee can push for the clean experiment—but I wouldn't cite it in its current form.\n\nRecommendation: send to review, but expect major revision; the authors need to establish that the mapping model generalizes to unseen texts.","headline":"Clever framework, but the headline numbers are likely inflated by training the surrogate on the very texts it later scores.","tokens_in":19682,"tokens_out":1989,"would_cite":false,"duration_ms":19957,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VeilProbe claims a fully automatic, text-only way to detect whether a document was part of an LLM's pre-training data, reporting AUC scores up to 0.963 and document-level AUC around 0.99.","keywords":["pre-training data detection","membership inference attack","black-box LLM","text-to-suffix mapping","prototype-based classifier","key token perturbation","information bottleneck","data contamination detection"],"falsifier":"Train the mapping model on text-to-suffix pairs from one random half of the candidate texts, evaluate the full VeilProbe pipeline on the other half, and compare sentence-level AUC and TPR@5%FPR with the paper's reported values; if the held-out numbers drop toward the grey-box baselines, the detection signal is memorization of the test texts rather than a transferable text-to-suffix rule.","tokens_in":18659,"feed_emoji":"🔍","tokens_out":10652,"duration_ms":91728,"temperature":0.7,"pith_summary":"The paper claims that pre-training membership can be detected in a black-box LLM, where the auditor sees only input text and generated output rather than token probabilities or internal parameters, and that this can be done without handcrafted prompts. Its framework, VeilProbe, asks the target LLM to continue each candidate text, trains a sequence-to-sequence model to imitate that text-to-suffix mapping, and uses the mapping model's hidden states plus calibration features from perturbed versions of the text as membership evidence. A prototype-based classifier trained on about 50 known training and non-training examples then decides membership. Across WikiMIA, BookTection, and arXivTection, the paper reports sentence-level AUC up to 0.963, TPR@5%FPR up to 0.845, and document-level detection averaging roughly 0.99 AUC. If true, an API-only auditor could screen corpora for pre-training membership at scale, with direct implications for copyright enforcement and privacy auditing.","feed_headline":"Text-only probe finds LLM pre-training data at 0.963 AUC","feed_subtitle":"VeilProbe uses only input and output text, beats probability-based detectors, and works on ChatGPT and Claude.","key_machinery":"The load-bearing object is the sequence-to-sequence mapping model: a transformer trained to maximize P(sout|s), whose hidden states are extracted as mapping features. Around it, key-token perturbation uses proxy LLMs with a feature-attribution method to choose the top-γ% tokens to replace with synonyms, producing calibration features Zc = Zs − Zr; a Welch significance test keeps only dimensions that separate known training from non-training texts; and a prototype-based classifier constructs class prototypes c0 and c1 with Information Bottleneck denoising, scoring each text by δ=d(Ẑ,c0)−d(Ẑ,c1). The machinery's job is to convert the target LLM's visible completion behavior into a fixed-dimensional feature that is more separable than the raw outputs themselves.","core_discovery":"The central claim is that a black-box detector can learn a membership signal from text-to-suffix behavior alone. For each candidate text s, VeilProbe obtains the target LLM's autoregressive continuation sout and a continuation rout for a synonym-perturbed version r of s; a transformer sequence-to-sequence model is trained on these pairs to capture the mapping f: s→sout, and its hidden states become the text-to-suffix mapping feature Zs. The difference Zs−Zr is filtered by a significance test to produce a perturbation calibration feature, and the two are concatenated into a membership feature Z. A prototypical network with an Information Bottleneck term turns Z into a detection score δ=d(Ẑ,c0)−d(Ẑ,c1) against learned pre-training and non-training prototypes. The paper reports sentence-level AUC of 0.902–0.963 on WikiMIA and BookTection, TPR@5%FPR up to 0.845, and document-level detection averaging roughly 0.99 AUC, exceeding both grey-box baselines that use token probabilities and black-box baselines that require human-designed tasks.","pith_inferences":["A strict inductive test, training the mapping model on one disjoint set of texts and evaluating on another, would determine whether the reported accuracy comes from a generalizable text-to-suffix rule or from the mapping model having already fit the candidate texts' continuation patterns; the paper's setup does not clearly separate these.","The use of proxy LLMs to choose perturbation tokens assumes the proxies' attribution preferences transfer to the target model; a target with very different tokenization, corpus, or generation style could lose the calibration feature's separating power.","The same feature-extraction and prototype-classification recipe could apply to related membership-detection tasks, such as data-contamination tests or fine-tuning-data detection, whenever an API exposes only completions.","If the claimed signal is real, it shows that output-only interfaces leak membership information, strengthening the case for output filtering or differential privacy in hosted models."],"forward_implications":["An API-only auditor could screen large text corpora for LLM pre-training membership without token-probability access, something grey-box methods cannot do for products like ChatGPT or Claude.","Document-level screening of books and papers would reach average AUC around 0.99 when sentence-level scores are aggregated, making copyright infringement screening practical at scale.","Because detection still works with only ten labeled ground-truth samples at roughly 80–90 percent AUC, deployment is feasible even when a provider discloses few or no training documents.","The framework is automated, so no per-text cloze questions or multiple-choice tasks need to be designed, and the same pipeline can be applied to new domains without human effort."],"supporting_citations":[{"why":"Supplies the WikiMIA dataset, the Min-K% Prob baseline, and the task formulation the paper builds on.","marker":"[Shi et al., 2024]"},{"why":"Provides the BookTection and arXivTection datasets and the DE-COP black-box baseline that VeilProbe compares against.","marker":"[Duarte et al., 2024]"},{"why":"Provides the Name-Cloze black-box baseline and the premise that popular books are known to be included in training corpora.","marker":"[Chang et al., 2023]"},{"why":"Supplies the prototypical-network training procedure and prototype construction used by the membership classifier.","marker":"[Snell et al., 2017]"},{"why":"Supplies the transformer architecture on which the sequence-to-sequence mapping model is built.","marker":"[Vaswani et al., 2023]"},{"why":"Supplies the GILOT feature-attribution method used to select key tokens for perturbation.","marker":"[Li et al., 2024]"},{"why":"Motivates the perturbation calibration feature through the differing robustness of models to perturbed training versus non-training inputs.","marker":"[Liu et al., 2024]"},{"why":"Provides the Information Bottleneck objective used to denoise membership features.","marker":"[Tishby and Zaslavsky, 2015]"},{"why":"Supplies the observation that hidden states of a white-box LLM separate training from non-training texts, motivating the mapping-feature approach.","marker":"[Wang et al., 2024]"}],"fun_headline_variants":["VeilProbe: black-box LLM pre-training text detection without humans","Pure text-to-suffix mapping exposes pre-training data in black-box LLMs","VeilProbe: automated black-box detector for LLM pre-training data","Black-box only: VeilProbe finds pre-training text with 0.96 AUC"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the mapping model that produces membership features has not been trained on the same candidate texts it later classifies, since the paper trains it on text-to-suffix pairs generated from the evaluation corpus and reserves only about 50 samples for the classifier.","fun_headline_variants_meta":{"raw":{"variants":["VeilProbe: black-box LLM pre-training text detection without humans","Pure text-to-suffix mapping exposes pre-training data in black-box LLMs","VeilProbe: automated black-box detector for LLM pre-training data","Black-box only: VeilProbe finds pre-training text with 0.96 AUC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000663,"raw_usage":{"total_tokens":3046,"prompt_tokens":979,"completion_tokens":2067,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":1981}},"tokens_in":595,"tokens_out":2067,"duration_ms":13577,"temperature":1.0,"reasoning_tokens":1981,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:07:13.780263+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the mapping model on text-to-suffix pairs from one random half of the candidate texts, evaluate the full VeilProbe pipeline on the other half, and compare sentence-level AUC and TPR@5%FPR with the paper's reported values; if the held-out numbers drop toward the grey-box baselines, the detection signal is memorization of the test texts rather than a transferable text-to-suffix rule.","supporting_citations":[{"cited_title":"Duarte, Xuandong Zhao, Ar- lindo L","cited_arxiv_id":null,"evidence_quote":"Provides the BookTection and arXivTection datasets and the DE-COP black-box baseline that VeilProbe compares against."},{"cited_title":"Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman","cited_arxiv_id":null,"evidence_quote":"Provides the Name-Cloze black-box baseline and the premise that popular books are known to be included in training corpora."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the prototypical-network training procedure and prototype construction used by the membership classifier."},{"cited_title":"Gomez, Lukasz Kaiser, and Illia Polosukhin","cited_arxiv_id":null,"evidence_quote":"Supplies the transformer architecture on which the sequence-to-sequence mapping model is built."},{"cited_title":"Please tell me more: Privacy impact of explain- ability through the lens of membership inference attack","cited_arxiv_id":null,"evidence_quote":"Motivates the perturbation calibration feature through the differing robustness of models to perturbed training versus non-training inputs."},{"cited_title":"Deep learning and the information bottleneck principle,","cited_arxiv_id":null,"evidence_quote":"Provides the Information Bottleneck objective used to denoise membership features."},{"cited_title":"Wang, Jason Wang, Marvin Li, and Seth Neel","cited_arxiv_id":null,"evidence_quote":"Supplies the observation that hidden states of a white-box LLM separate training from non-training texts, motivating the mapping-feature approach."}],"review_version":1}