{"id":"7df15a2c-5951-4ab0-b533-3ac337fb3d29","arxiv_id":"2502.09974","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A permutation test on BERT embeddings of LLM outputs can detect, with statistical significance, when response distributions differ because a chat service uses a different system prompt than a candidate prompt.","lead":"This paper introduces Prompt Detective, a statistical method that checks whether a third-party chatbot uses a given system prompt by comparing the distributions of its replies with those produced by a known model using the candidate prompt. The method detects prompt differences with statistical tests, but it is most reliable when the target model is known in advance and the system prompt is not identical.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Prompt Detective's permutation test can reject equality of output distributions, but a non-significant p-value is not positive evidence that the prompt was reused; the headline 'verify prompt usage' claim outruns the statistical procedure.","rationale":"The reader's formal weakest_assumption is exchangeability of outputs when prompts are identical (same model version, chat template, decoding settings, and no hidden instructions). That is a real external-validity threat, and the reader's rationale also explicitly notes that 'a non-significant result is not positive evidence for reuse.' My read goes one step further: the logical inversion is internal to Algorithm 1 and holds even when all exchangeability assumptions are satisfied. The permutation test is calibrated to reject equality; it cannot accept equality. Therefore the central claim that Prompt Detective 'verifies prompt usage with statistical significance' is not supported by the experiment design. However, the paper does establish a narrower and still useful finding: with query access, a known reference model, and enough samples, the test reliably separates output distributions induced by different system prompts, including near-identical rephrasings. That finding is credible and the code is available for verification. Because the gap is between the statistical decision rule and the headline interpretation, and because it is addressable by reframing the contribution and adding an equivalence-based decision rule or a formal power analysis, the reader's CONDITIONAL verdict remains appropriate. I therefore leave the verdict unchanged and make the required conditions more specific.","tokens_in":15004,"tokens_out":15493,"duration_ms":157790,"concrete_test":"Re-analyze the hard-example data (Similarity Levels 1-3) with an equivalence test: pre-specify an equivalence margin delta on the difference of mean BERT embeddings (or on 1-cosine similarity), compute a 90% confidence interval for each pair, and classify the prompt as 'reused' only if the interval lies entirely within [-delta, delta]. Report the true reuse rate on the same-prompt pairs and the false reuse rate on the Similarity Level 1-3 different-prompt pairs. If the false reuse rate exceeds the nominal level, or if no delta yields both high true reuse and controlled false reuse, the current p-value-based decision rule cannot verify prompt reuse.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 tests the null that the two output distributions are equal and returns 'Prompts are distinct' (p<alpha) or 'Insufficient evidence to claim prompts are distinct' (p>=alpha). The abstract and Figure 1 interpret the second outcome as 'Proprietary Prompt has been reused.' That inference is invalid: a non-significant p-value does not establish the null. Under the null, p-values are uniform, so p>=0.05 occurs with probability 0.95 for identical prompts, but it also occurs for different prompts when power is insufficient. Table 2 shows the practical consequence: with k=2 generations per task prompt, Similarity Level 1 pairs (known different prompts) have average p-values of 0.19-0.31 and reported error rates around 0.65 for Claude Haiku and GPT-3.5, so 'p>=0.05 => reused' has no controlled false-reuse rate, even under perfect exchangeability. To verify reuse one needs an equivalence test with a pre-specified margin or a power analysis covering all relevant alternative prompts. The experiments support only a narrower claim: Prompt Detective detects when the deployed prompt differs measurably from the known prompt; they do not support positive verification of reuse.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"Prompt Detective is a training-free statistical method for prompt membership inference. Given query access to a third-party chat service and a candidate proprietary system prompt, it generates k responses to n task prompts from both the service and a reference model configured with the known prompt, embeds all outputs with BERT, and applies a block permutation test using cosine similarity between mean embeddings as the test statistic. The paper reports that at a 0.05 significance threshold the test separates identical from different prompts in standard and hard-example settings, including near-identical rephrasings and typo-level differences, and extends the method to a black-box scenario with a small candidate set of six models.","tokens_in":15196,"tokens_out":8222,"duration_ms":82537,"significance":"If the central claim were established, this would be a useful contribution to prompt privacy: a cheap, training-free audit that detects when a suspected prompt differs from the prompt actually deployed in a chat service. The method has real strengths: it uses no fitted parameters, relies on a standard permutation test on fixed embeddings, ships code, and is evaluated across six model families and against a PLeak baseline. However, the paper's headline claim of positive verification of prompt reuse is not supported by the statistical procedure; the controlled experiments establish at most that the test can detect distributional differences under favorable conditions. The logical gap is fixable, but it requires reworking the decision rule, the claims, and the black-box evaluation.","major_comments":[{"comment":"Algorithm 1 tests the null hypothesis that the two output distributions are equal and returns 'Prompts are distinct' only when p<α; the complementary outcome is 'Insufficient evidence to claim prompts are distinct.' A non-significant p-value does not establish that the null is true, so the interpretation in the abstract ('verify prompt usage'), Figure 1 ('Proprietary Prompt has been reused'), and Section 7 is logically invalid. Under the null, p-values are uniform, so p≥0.05 occurs with probability 0.95 for identical prompts, but it also occurs for different prompts whenever power is insufficient. Table 2 shows the practical consequence: for Similarity Level 1 pairs, which are known different prompts, Claude2 and GPT-3.52 have average p-values of 0.194 and 0.213 and reported false-positive rates of 0.65; a rule that maps p≥0.05 to 'reused' would therefore declare reuse in 65% of known non-reuse cases. To support positive verification, the authors need an equivalence test with a pre-specified margin, or a power analysis that covers all plausible alternative prompts and model configurations.","section":"Abstract; Fig. 1; Algorithm 1; §5.1"},{"comment":"The permutation test's null distribution is only valid if, when the candidate and deployed system prompts are identical, the third-party model's outputs are exchangeable with the reference model's outputs within each task prompt. This requires the same model weights or version, chat template, decoding temperature, sampling settings, and no additional hidden system-level instructions. Section 3.1 assumes knowledge of the deployed model 'in most of our experiments,' and Section 6 relaxes this only to a candidate set of six known models. If any of these conditions fails, the test can reject even when the same system prompt is in use, so the reported FPRs in Table 1 do not transfer to real third-party services. A concrete experiment holding the system prompt fixed while varying chat template, temperature, or model version is needed before the verification claim can be assessed.","section":"§3.1, §6; Algorithm 1"},{"comment":"The black-box procedure that takes the maximum p-value over the six reference models and applies a Bonferroni correction does not define a valid test for the composite question 'is the unknown deployed model, among the candidates, using pbar?' Taking a maximum p-value makes the decision 'prompts distinct' conservative in one direction, but it does not control the error of declaring 'reused' when the deployed prompt is different but produces outputs close to one of the reference distributions. Table 3 is evaluated under the assumption that the true model is in the candidate set, and the FPR/FNR numbers therefore do not quantify the false reuse rate in the setting the abstract advertises. The authors should either formulate a single composite test with a well-defined null and report its power, or restrict all claims to the known-model case.","section":"§6, Table 3"}],"minor_comments":[{"comment":"The meaning of FPR is inconsistent with Table 1. In Table 1, FPR is the type-I error of rejecting equality for identical prompts, whereas Table 2 rows are known different prompts, so FPR must be a different quantity. Please define both quantities explicitly in each caption.","section":"Table 2, caption"},{"comment":"If no permuted similarity is as extreme as the observed value, the reported p-value is 0, which is not a valid finite-sample estimate. Use (c+1)/(Npermutations+1) or state that Npermutations is large enough to make this correction negligible.","section":"Algorithm 1, Step 5"},{"comment":"The caption says generations from different but conceptually similar prompts 'cluster together,' while the text and figure appear to show separate clusters. This likely should read 'cluster separately.'","section":"Figure 5, caption"},{"comment":"The quantities pp_avg and pn_avg are used without definitions. They should be defined as the average p-values for positive (identical) and negative (different) prompt pairs, respectively.","section":"Table 1, caption"},{"comment":"All task prompts and hard-example variants are generated by Claude 3 Sonnet, and no ablation varies the probe-generation model. The reported performance may depend on this choice, so the paper should either test this dependence or discuss the limitation explicitly.","section":"Appendix F"}],"recommendation":"major_revision","confidential_remarks":"The paper's main flaw is the direction of inference: a non-significant p-value is treated as positive evidence of prompt reuse. This is a load-bearing logical error in the abstract, Figure 1, and Section 7, and it is not resolved by the black-box extension. The method itself appears sound for detecting distributional differences in controlled settings, and the empirical study is fairly thorough. If the authors reframe the contribution as a difference-detection test and add equivalence testing or a power analysis to justify any positive verification claim, the paper could become publishable. I would also recommend that the editor ask for a clarification of the FPR definitions in Tables 1 and 2, since the current labeling obscures the central issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Roman, you should know two things about this paper. First, the core method is real: framing prompt-usage verification as membership inference and running a block permutation test on BERT embeddings is a clean, training-free idea, and the controlled experiments show it works. Second, the paper's main claim is overreached. A non-significant p-value is not positive evidence that the prompt was reused, and the authors' own Table 2 demonstrates the problem. With k=2 generations per task prompt, different-but-similar prompts give p>=0.05 most of the time, so interpreting 'insufficient evidence' as 'reused' would produce garbage. The algorithm box is careful — it says 'insufficient evidence to claim prompts are distinct' — but the abstract and Figure 1 say 'has been reused.' That is a load-bearing statistical error, not a wording nit.\n\nWhat is genuinely new: the problem formulation itself. Prompt extraction and data membership inference are both well-studied, but prompt membership inference as a query-only statistical test is not in the cited literature. The method is simple, has no trained parameters, and the paper ships code and data. The experiments are thorough: six model families, two prompt sources, hard examples with graded similarity, a black-box candidate-set variant, an embedding ablation, and a comparison against PLeak. The hard-example results with 50 generations per task prompt are impressive — near-perfect separation even for minimal rephrasing — and the finding that minor prompt changes leave detectable traces in output distributions is a legitimate empirical contribution.\n\nThe soft spots beyond the logical one are real but minor in comparison. The permutation test's validity requires exchangeability, which means the same underlying model, same chat template, same decoding settings, and no hidden extra instructions. The paper assumes model knowledge in most experiments and only relaxes this to a small candidate set. So in practice the test detects distribution mismatch, not necessarily prompt reuse. Also, Table 2's 'FPR' column for similar-but-different prompts is a misnomer; those are negative pairs, so it is not a false positive rate. That naming confused my reading and should be fixed.\n\nAll told, this paper deserves a serious referee. The problem is timely, the method is reproducible, and the statistical overreach is fixable with an equivalence test or a power analysis that covers relevant alternative prompts. I would send it to review with a clear request to revise the claim and the figure, but I would not desk-reject it. The core capability is real and worth publishing once the decision rule is made honest.","headline":"Prompt Detective is a genuinely new, simple, and well-tested method for detecting when a system prompt produces measurably different output distributions, but the paper's headline claim that it 'verifies prompt usage' outruns the statistics.","tokens_in":15754,"tokens_out":1758,"would_cite":true,"duration_ms":19820,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Prompt Detective, a training-free statistical test, can verify whether a given system prompt was used by a third-party language model by comparing the distributions of their responses.","keywords":["prompt membership inference","system prompt verification","permutation test","large language models","prompt reuse detection","statistical hypothesis testing","BERT embeddings","prompt privacy"],"falsifier":"Take one system prompt, run it in two deployments that use different chat templates or different sampling temperatures but the same visible prompt, and apply Prompt Detective with enough responses; if it returns p < 0.05, the test is detecting differences in conditioning rather than prompt reuse specifically, which would disprove the claim that statistical rejection identifies prompt membership.","tokens_in":14752,"feed_emoji":"🔍","tokens_out":7807,"duration_ms":72100,"temperature":0.7,"pith_summary":"Prompt Detective is a training-free statistical test for deciding whether a given system prompt was actually used inside a third-party chatbot. It queries the suspect service and a reference model with the same task prompts, embeds the responses as vector representations, and runs a permutation test on the cosine similarity between the group means. The paper shows that across several large language models and prompt collections, the test reliably rejects the null hypothesis of equal output distributions whenever the system prompts differ, even for minimal rephrasing or a two-character typo. If the claim holds, prompt reuse can be verified with statistical significance rather than by attempting costly and imperfect prompt reconstruction.","feed_headline":"Statistical test tells whether a given system prompt is in use","feed_subtitle":"Prompt Detective compares output distributions to verify prompt reuse, even detecting paraphrases and typos.","key_machinery":"The load-bearing object is a block permutation test on a cosine-similarity statistic. Each generated response is mapped to a vector by a sentence-embedding model; the statistic is the cosine similarity between the mean vectors of the two groups. To build a null distribution, responses are shuffled between the two groups within each task prompt, preserving the block structure that responses to the same query are more alike. The p-value is the fraction of permuted similarities at least as extreme as the observed one, yielding a non-parametric test of equality of output distributions without training anything.","core_discovery":"On the paper's own terms: even very small changes to a system prompt leave a measurable fingerprint in the distribution of a language model's responses. Prompt Detective exploits this by comparing two groups of generations, one from the third-party service and one from a reference model running the known proprietary prompt, and applying a block permutation test to the mean cosine similarity of their embeddings. In the positive case, where the same prompt sits on both sides, the test's p-values stay near 0.5; in the negative case they collapse toward zero once enough task prompts and generations are collected. The paper reports zero or near-zero false positive rates in standard settings, shows that 50 generations per task prompt separate even the most similar rewordings, and demonstrates that the method degrades gracefully when the underlying model is unknown but restricted to a small candidate set.","pith_inferences":["(Editorial inference) Because the test rejects on any systematic difference in response distributions, it could be repurposed to audit hidden instructions or drift in a deployed system prompt over time, not just to verify a known prompt.","(Editorial inference) The sensitivity to decoding settings implies that a clean negative result is also evidence that the model version and sampling procedure are stable, which matters if the method is used for forensic or legal claims.","(Editorial inference) The same block-permutation logic could be ported to other output representations or even non-text modalities, likely requiring more samples when the embedding is less sensitive to prompt differences."],"forward_implications":["A service owner can verify suspected prompt theft by querying the chatbot and comparing output distributions, without ever recovering the prompt text.","Prompts that differ by only a few words or a typo are still distinguishable, so paraphrase-based evasion would not survive once enough responses are collected.","The number of responses, not their length, drives statistical power: the paper finds 300 short responses are enough to separate the hardest pairs, and longer generations add little.","The method works even when the underlying model is unknown, provided the set of plausible models is small, after applying a multiple-comparison correction.","The tailored statistical test substantially outperforms adapting a prompt-reconstruction method to the verification task."],"supporting_citations":[{"why":"Supplies the permutation-test methodology that Prompt Detective uses for its statistical decision.","marker":"Good (2013)"},{"why":"Provides the sentence-embedding model that maps generated responses to fixed-size vectors.","marker":"Reimers & Gurevych (2019)"},{"why":"The prompt-reconstruction method used as the baseline in the membership-inference comparison.","marker":"Hui et al. (2024)"},{"why":"Frames membership inference against machine-learning models, the adversarial lens the paper adapts to prompts.","marker":"Shokri et al. (2017)"},{"why":"Provides the first-principles view of membership inference that motivates the output-distribution approach.","marker":"Carlini et al. (2022)"},{"why":"Supplies the Llama 2 model used in the experiments.","marker":"Touvron et al. (2023)"},{"why":"Provides the Claude 3 model family and the prompt library used as an experimental source.","marker":"Anthropic (2024a)"}],"fun_headline_variants":["LLM prompts leave fingerprints that reveal reuse","Statistical test detects if your prompt is used by an LLM","Prompt Detective: verify prompt use via response distributions","Tiny prompt tweaks still leak detectable patterns in LLM output","Prompt fingerprints: statistical test catches LLM prompt reuse"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The test assumes that when the candidate and target system prompts are identical, the third-party model's outputs and the reference model's outputs are exchangeable within each task prompt, which requires the same underlying model, chat template, decoding settings, and no hidden extra conditioning.","fun_headline_variants_meta":{"raw":{"variants":["LLM prompts leave fingerprints that reveal reuse","Statistical test detects if your prompt is used by an LLM","Prompt Detective: verify prompt use via response distributions","Tiny prompt tweaks still leak detectable patterns in LLM output","Prompt fingerprints: statistical test catches LLM prompt reuse"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000153,"raw_usage":{"total_tokens":1150,"prompt_tokens":832,"completion_tokens":318,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":448,"completion_tokens_details":{"reasoning_tokens":240}},"tokens_in":448,"tokens_out":318,"duration_ms":3912,"temperature":1.0,"reasoning_tokens":240,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T19:51:28.188677+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one system prompt, run it in two deployments that use different chat templates or different sampling temperatures but the same visible prompt, and apply Prompt Detective with enough responses; if it returns p < 0.05, the test is detecting differences in conditioning rather than prompt reuse specifically, which would disprove the claim that statistical rejection identifies prompt membership.","supporting_citations":[],"review_version":1}