{"id":"dcc8977d-468a-47c4-8d67-8203ff69e2ca","arxiv_id":"2507.04478","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Simple prompts to Llama 3.2 1B produce text that resembles PII, but the paper does not establish that this text is memorized training data.","lead":"This paper tries to extract personal information from Meta's Llama 3.2 1B by feeding it simple prompts like 'my password is:'. It reports three generated fragments as proof that the model leaks memorized passwords, emails, and account numbers.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim equates PII-shaped output with memorized training data; Table I shows only generic placeholders and Section V's Google-search validation cannot establish memorization, so the extraction result is unsupported.","rationale":"The reader's weakest assumption is exactly the load-bearing weakness: the inference from PII-shaped text to memorized training strings. My reading finds no independent support for that inference. The visible outputs are not convincingly PII, the redacted table prevents verification, and Section V's validation only confirms existence, not provenance. The paper's own Section VIII concedes the absence of memorization rates. Since the central claim 'we extracted PII' is unsupported, the rejection stands. I would add one nuance: the paper's mitigation discussion is reasonable but does not rescue the empirical claim. The proposed concrete test would settle the issue quickly because the model and prompts are fully specified.","tokens_in":3699,"tokens_out":2775,"duration_ms":29069,"concrete_test":"Run the paper's provided code with the exact prompts ('my password is:', 'account number:', 'my email id:') and record the full, unredacted outputs. Then apply a membership test: compute the model's per-token perplexity on each generated sequence and compare against the distribution over random continuations, and search for exact quoted substrings in a large public corpus (e.g., Common Crawl or the Llama 3.2 training data if accessible). If the outputs are high-perplexity generic strings or do not match any training document, the memorization claim is falsified. Additionally, run 1,000 control prompts with the same decoding parameters; if PII-like outputs occur at comparable rates for generic prompts, the extraction claim is an artifact of prompting, not memorization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the inference in Section V that a prompt-completion resembling PII is a copy of a memorized training string. This step fails for three reasons. (1) The reported outputs in Table I are not actual PII: 'my password is: 1a' is a generic fragment; 'account number:' yields no account number, only a bank name and PO Box; 'my email id:' yields 'vishal @gmail.com' with a space, a placeholder-like string. (2) Google-search validation only shows that a real Suncorp Bank and a LinkedIn user exist; it does not show the generated token sequence appears in Llama 3.2's training corpus, nor does it distinguish memorization from plausible hallucination. (3) The paper cites Carlini et al. but applies none of the membership or extraction tests used there (e.g., perplexity thresholds, exact-match criteria, canary audits). Equation (2) is defined but never instantiated with counts. Therefore the abstract's claim that passwords, emails, and account numbers were extracted is not supported by the evidence presented. A hallucinated email-shaped string would be expected from these generic prompts even in a model with no memorized PII.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper claims to demonstrate a black-box model inversion attack on Llama 3.2 1B, in which prompts such as 'my password is:', 'account number:', and 'my email id:' elicit memorized personally identifiable information (PII). The evidence consists of one redacted table (Table I) containing three generated fragments, a brief description of Google-search validation in Section V, and a discussion of mitigation strategies. The paper defines a memorization rate in Equation (2) but does not compute it, and Section VIII explicitly concedes that no specific memorization rates for Llama 3.2 1B are reported. The central claim that passwords, email addresses, and account numbers were extracted is therefore unsupported by the presented measurements.","tokens_in":4048,"tokens_out":2437,"duration_ms":26461,"significance":"If the central claim were established, it would be a meaningful privacy finding for a small open-source model. However, as presented, the paper provides no valid evidence of memorization: the Table I outputs are generic or truncated fragments, the Google-search validation cannot distinguish memorization from hallucination, and no membership test, likelihood threshold, or exact-match criterion is applied. The paper correctly cites the standard autoregressive factorization (Eq. 1) and prior extraction work by Carlini et al. and Nasr et al., but it does not carry over any of their measurement or verification methodology, so the result does not advance the state of the art. The reproducible code snippet is a minor positive, but no data or full outputs are released.","major_comments":[{"comment":"The reported 'extracted PII' does not support the abstract's claim that passwords, email addresses, and account numbers were extracted. 'my password is: 1a' is not a password; 'account number: , name: Suncorp Bank, address: PO Box 1444' contains no account number; and 'my email id: vishal @gmail.com, my mobile no: 97' contains a space before '@gmail.com' and a truncated phone number. These outputs are consistent with generic placeholder-like generation and do not constitute PII extraction.","section":"Section IV, Table I"},{"comment":"The Google-search validation cannot establish that the generated sequences were memorized training data. Showing that Suncorp Bank and a LinkedIn user exist does not demonstrate that the exact generated token sequence appears in Llama 3.2's training corpus, nor does it rule out the possibility that the model hallucinated an email-shaped or bank-shaped string. The paper cites Carlini et al. but applies none of the membership or extraction tests used there, such as perplexity thresholds, exact-match criteria, or canary audits, so the inference from 'output resembles a real entity' to 'memorized training data' is not supported.","section":"Section V"},{"comment":"Equation (2) defines a memorization rate, but the paper never instantiates it with any counts. The claim in Section VI that 'Some experiments yielded a non-zero memorization rate' is unsupported by any data, and Section VIII explicitly concedes 'the lack of specific memorization rates for Llama 3.2 1B.' Without this quantitative measure, the paper's central assertion that the attack 'successfully extracted' PII cannot be evaluated.","section":"Section VI, Eq. (2)"},{"comment":"The methodology does not include any control or baseline to distinguish memorization from hallucination. Equation (1) is merely the standard autoregressive factorization of the probability of generating a sequence; it does not provide a test for whether a generated string was present in the training data. A model with no memorized PII would be expected to produce email-shaped and account-shaped completions from these generic prompts, so the observed outputs carry no evidentiary weight without additional membership or likelihood analysis.","section":"Section III"}],"minor_comments":[{"comment":"In the Introduction, 'focuses on the Llama 3.2 an LLM developed by Meta' is missing the word 'is' or a comma before 'an LLM.'","section":"Section I"},{"comment":"There is a missing space in 'Model inversion attacks(MIA)'; it should read 'attacks (MIA).'","section":"Section II.A"},{"comment":"Reference [7] is cited for the claim that 'requiring an API key or solving CAPTCHAs can prevent automated mass extraction,' but [7] is the OWASP API Security Top 10; the citation does not appear to support this specific claim.","section":"Section VII"},{"comment":"Reference [8] is cited for the statement that 'Google and Apple have used DP in production language models to protect user data,' but reference [8] is a 2007 paper on stopping spam by solving puzzles, which does not support this claim.","section":"Section VII"},{"comment":"The caption states that outputs are 'analyzed (e.g., via likelihood),' but no likelihood analysis is performed anywhere in the paper; either add such an analysis or revise the caption.","section":"Figure 1"}],"recommendation":"reject","confidential_remarks":"The manuscript is far below the evidentiary standard for a journal paper: the central empirical claim rests on three redacted, non-informative outputs and a Google search that cannot establish memorization. The paper also contains multiple citation mismatches in the mitigation section. I see no path, within the current scope, to repair the missing measurements; the authors would need to run proper extraction/evaluation experiments and report full outputs and membership tests."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this one doesn't support its headline. The paper claims to extract PII from Llama 3.2 1B, but the evidence is three hand-picked prompts and generated fragments that are mostly placeholders. Nothing separates memorized training data from a model's ordinary ability to produce plausible-looking strings.\n\nWhat's worth credit: the target is modestly interesting. Prior extraction results were mostly on larger models, and showing what a 1B edge model does is a legitimate small data point. The code snippet is reproducible, the paper is clearly organized, and Section VIII honestly notes that no memorization rates were computed. The discussion of mitigations is generic but not wrong.\n\nThe problems are load-bearing. Table I's outputs are redacted, but the visible parts are not impressive: 'my password is: 1a', a bank name and PO Box, 'vishal @gmail.com' with a space. These are exactly the kind of generic completions an LM would produce from those prompts. The Google-search validation in Section V only shows that Suncorp Bank and a LinkedIn profile exist; it doesn't show that the exact token sequence appears in the training corpus. Equation (2) defines a memorization rate but it is never computed. The paper cites Carlini et al. and Nasr et al. but applies none of their membership or extraction tests, such as perplexity thresholds or canary audits. Without that, the conclusion in the abstract and Section VI is an overreach.\n\nThe paper even admits this in Section VIII: 'Limitations include the lack of specific memorization rates for Llama 3.2 1B.' That sentence, read carefully, concedes the central measurement is missing.\n\nWho is this for? Maybe someone wanting a minimal example of prompting a small model for PII-shaped output, but as a research contribution it's not there. It deserves a desk reject, not referee time. If the author wants to salvage it, they'd need to run a proper extraction benchmark with controls, exact-match or likelihood-based membership tests, and ideally compare against known baselines.\n\nRecommendation: don't send to peer review. Not worth a serious referee's hours.\n\nRegards,","headline":"A routine prompting demo that overclaims extraction: without a membership or likelihood test and with the memorization rate never computed, the central result is unsupported.","tokens_in":4446,"tokens_out":2748,"would_cite":false,"duration_ms":27920,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that short templated prompts can extract memorized personal data from the Llama 3.2 1B model through black-box queries.","keywords":["model inversion attack","large language models","PII extraction","memorization","privacy leakage","Llama 3.2","black-box attack","prompt-based extraction"],"falsifier":"Check whether the exact generated strings (for example 'vishal@gmail.com' and 'PO Box 1444') appear anywhere in Llama 3.2's pretraining data; if they are absent, the outputs are plausible hallucinations rather than recovered training records. A complementary test would be to run the same prompts on a version of the model trained without those strings and see whether it emits the same completions.","tokens_in":3487,"feed_emoji":"🕵️","tokens_out":6157,"duration_ms":60920,"temperature":0.7,"pith_summary":"This paper claims that Llama 3.2 1B, a small open-weight language model, can be induced through short black-box prompts such as \"my password is:\" or \"account number:\" to emit strings that resemble real people's emails, phone numbers, and account details. The author presents these emitted strings as proof that the model has memorized personally identifiable information from its training data, and concludes that even small edge-deployed models are vulnerable to model inversion attacks. The significance, if true, is that any public or API-accessible LLM could be probed for sensitive records without any internal access. The paper also proposes standard defenses—query limiting, differential privacy, data sanitization, output filtering, and auditing—though it does not measure their effectiveness.","feed_headline":"Templated prompts extract PII from Llama 3.2 1B, paper says","feed_subtitle":"If true, even small open-weight models can be probed for memorized emails, phone numbers, and account details.","key_machinery":"The mechanism that carries the attack is the autoregressive text-generation loop: a short, templated prompt is passed to the model through the Hugging Face Transformers pipeline, and the model continues the sequence. The extraction probability is formalized in the paper as a chain-rule product of conditional token probabilities, linking successful extraction to the model's softmax distribution. The paper's \"Memorization Rate\" is defined to quantify how often such extractions succeed per query, although no actual rate for the 1B model is reported.","core_discovery":"The central discovery the paper reports is that a black-box model inversion attack on Llama 3.2 1B succeeds: by feeding the model templated prompts like \"my password is:\", \"account number:\", and \"my email id:\", the author obtained completions that include a personal email address, a phone number, a name, a LinkedIn URL, and a bank account holder's name and P.O. Box address. The paper models the extraction probability as a product of per-token conditional probabilities (Equation 1) and reports a non-zero memorization rate defined as extracted PII sequences divided by total queries. The outputs were validated by Google searching the LinkedIn URL and the bank address, which the author takes as evidence that the strings were memorized during training.","pith_inferences":["The evidence presented does not rule out the null hypothesis that the model is hallucinating email- and account-shaped text: the Google-search check confirms that the outputs resemble real entities, but not that they are memorized copies. A membership-inference test comparing the likelihood of the exact string against a control string would settle this.","If these extractions are genuine memorization, the trigger is almost certainly duplicated data in the pretraining corpus; measuring the frequency of the extracted strings across web corpora would predict attack success and connect this result to known deduplication defenses.","The same prompt templates could be run across other small open-weight models to produce a memorization benchmark; the paper's handful of examples is not enough to estimate a memorization rate for Llama 3.2 1B, as the author concedes in the limitations.","One testable extension is to measure how the extraction success rate varies with temperature and top-k; the paper fixed these to high-likelihood values (top-k=40, top-p=1), so the reported vulnerability may shrink under more diverse decoding settings."],"forward_implications":["If the attack works as reported, anyone with API access to Llama 3.2 1B can probe for memorized PII with short, templated prompts and no model internals.","A non-zero memorization rate for the 1B model would mean that smaller, edge-deployed models are not inherently safe from inversion attacks, contrary to the assumption that only large models leak.","The paper's proposed defenses—rate limiting, output filtering, differential privacy, and data sanitization—would all become relevant for production use of open-weight models, though access control offers no protection once weights are public.","The extraction probability model in Equation (1) implies that attack success could be amplified by high-likelihood decoding settings, which the paper uses by design, so real-world risk depends on the attacker's choice of sampling parameters."],"supporting_citations":[{"why":"Supplies the attack methodology and the comparative baseline memorization rate (0.789% for LLaMA 65B) that the paper claims its non-zero rate echoes.","marker":"[3]"},{"why":"Defines model inversion attacks and motivates why querying outputs can recover training data.","marker":"[2]"},{"why":"Provides the target model's identity, architecture, and training process that the paper assumes may include sensitive data.","marker":"[5]"},{"why":"Supplies prior evidence that high-likelihood outputs can flag memorized sequences, used in the paper's auditing recommendation.","marker":"[1]"},{"why":"Supports the claim that repeated or deduplicated snippets amplify memorization, which motivates the data-sanitization defense.","marker":"[13]"}],"fun_headline_variants":["Black-box prompts pull passwords and emails out of Llama 3.2","Llama 3.2 1B leaks PII via templated inversion attack","Model inversion attack extracts passwords and addresses from Llama 3.2","Simple prompts force Llama 3.2 1B to spill memorized PII","Templated queries expose private data in Llama 3.2 1B"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's conclusion assumes that any generated completion that looks like PII is a verbatim copy of a memorized training string; the Google-search validation only shows the output resembles a real person or bank, not that those exact tokens appear in the training corpus.","fun_headline_variants_meta":{"raw":{"variants":["Black-box prompts pull passwords and emails out of Llama 3.2","Llama 3.2 1B leaks PII via templated inversion attack","Model inversion attack extracts passwords and addresses from Llama 3.2","Simple prompts force Llama 3.2 1B to spill memorized PII","Templated queries expose private data in Llama 3.2 1B"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000179,"raw_usage":{"total_tokens":1233,"prompt_tokens":811,"completion_tokens":422,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":427,"completion_tokens_details":{"reasoning_tokens":315}},"tokens_in":427,"tokens_out":422,"duration_ms":4567,"temperature":1.0,"reasoning_tokens":315,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:45:59.837307+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check whether the exact generated strings (for example 'vishal@gmail.com' and 'PO Box 1444') appear anywhere in Llama 3.2's pretraining data; if they are absent, the outputs are plausible hallucinations rather than recovered training records. A complementary test would be to run the same prompts on a version of the model trained without those strings and see whether it emits the same completions.","supporting_citations":[{"cited_title":"Llama 3.2: A Lightweight Multilingual Language Model,","cited_arxiv_id":null,"evidence_quote":"Provides the target model's identity, architecture, and training process that the paper assumes may include sensitive data."},{"cited_title":"Statistics of noninteracting many-body fermionic states: The question of a many-body mobility edge","cited_arxiv_id":"2306.11692","evidence_quote":"Supports the claim that repeated or deduplicated snippets amplify memorization, which motivates the data-sanitization defense."}],"review_version":1}