{"id":"24e07591-6162-4556-8721-66ceebe893ea","arxiv_id":"2506.17279","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Step-by-step reasoning prompts can recover purportedly erased facts from unlearned LLMs, but the paper's quantitative evidence is internally inconsistent.","lead":"This paper introduces SLEEK, a black-box attack that uses step-by-step reasoning prompts to recover knowledge that LLM unlearning methods claim to erase. It reports leakage rates across four unlearning methods and two LLMs, but key numbers contradict the paper's own tables.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The attack's success metric, keyword presence in outputs, counts denials and hallucinations as leakage; the paper's own WHP examples show this inflates the reported rates.","rationale":"The reader identified keyword presence as the weakest assumption; the manuscript's own examples make the failure concrete, so I agree. This concern is load-bearing because every quantitative success rate in the paper's central evaluation depends on it. Even if all other inconsistencies (missing RMU table, UNSTAR table for Llama, absent artifacts) were fixed, the numbers would still not show knowledge recovery unless response-level annotation verifies that keywords correspond to facts. The internal mismatch between the 62.5% claim and Table 3 is additional evidence of reporting unreliability but secondary. A focused re-annotation experiment would settle whether SLEEK's success rates measure leakage or lexical adjacency. Since the current evidence does not substantiate the claim as written, the reader's REJECT verdict should stand.","tokens_in":13188,"tokens_out":3548,"duration_ms":41368,"concrete_test":"Use the original WHP-unlearned Llama responses from the Table 3 experiments. Have three blinded annotators label each response as (a) asserts the target fact, (b) mentions associated keywords but does not assert the fact, or (c) denies/refuses/hallucinates. Recompute the Direct/Implied/Indirect success rates using only category (a) responses as successful leakage, and separately verify whether any 62.5% figure corresponds to the forget set. If the corrected forget-set rates are materially lower than 27.5/37.5/40, or if no response asserts the fact, the paper's headline leakage claim is unsupported by its own metric.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 2 (Step 3) scores Direct and Indirect responses by counting keywords from a human-curated list (Appendix A.1: 'Hogwarts', 'Voldemort', 'Gryffindor', etc.) in the unlearned model's output. That metric cannot tell whether the model actually knows the forgotten fact. The paper's own WHP example shows the failure mode: asked 'What is the name of the school where Harry Potter was sorted?', the unlearned model responds 'I don't have information on a school called \"Hogwarts\"... The Harry Potter series is a fictional series of books created by Michael Coleman, and it does not include a school called \"Hogwarts.\"' A keyword counter records multiple hits ('Hogwarts', 'Harry Potter') and labels this an attack success, yet the response neither retrieves 'Hogwarts is Harry's school' nor demonstrates retained knowledge—it may even be a hallucination. The human-in-the-loop validation in Section 4.1 filters generated questions, not responses; the GPT score is applied only to Implied questions. Consequently the quantitative foundation of the central claim—the 62.5% figure and the Direct/Indirect rates in Tables 1–3—is not established. The mismatch between the abstract's 62.5% 'retrieved forgotten facts from WHP-unlearned Llama' and Table 3, where the only 62.5% cell is the WHP retain-set Direct rate, underscores that the reported numbers are not yet reliable evidence of knowledge recovery.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SLEEK, a black-box attack that uses step-by-step reasoning to probe LLMs after machine unlearning. The attack generates adversarial questions from a support LLM's chain-of-thought decomposition of forget-set and retain-set facts, clusters and human-filters them, and then evaluates the unlearned model's responses using keyword presence and a GPT-based score. The authors report high success rates, e.g., 62.5% retrieval of Harry Potter facts from WHP-unlearned Llama, and conclude that current unlearning methods fail to reliably erase knowledge and may unfairly suppress retained knowledge.","tokens_in":13383,"tokens_out":6974,"duration_ms":80540,"significance":"If the central claim were established, the work would be relevant to the LLM unlearning community: a systematic black-box attack that recovers supposedly erased knowledge across multiple unlearning methods and two LLMs would sharpen evaluation standards and highlight the limits of current suppression-based unlearning. The paper is clearly written, the attack pipeline is described in sufficient detail to be reproduced, and the authors make a genuine effort to include human-in-the-loop validation of generated questions, iterative expansion, and comparisons across four unlearning methods. However, the evaluation metric is not a valid measure of knowledge retention, the headline numbers are internally inconsistent, and the authors' own Limitations section (page 17) concedes that the analysis relies on specific domains and that human oversight introduces subjectivity. As presented, the quantitative evidence does not support the paper's central claims.","major_comments":[{"comment":"The Keyword Presence Score in Algorithm 2 (Step 3) labels a response as a successful retrieval whenever a keyword from a human-curated list (Appendix A.1) appears. The WHP example in §5.2 is a direct counterexample: the response \"I don't have information on a school called 'Hogwarts'...\" contains both \"Hogwarts\" and \"Harry Potter\" and would be scored as a Direct hit, yet the response explicitly denies the fact and even confabulates the author as \"Michael Coleman\". The metric therefore measures surface-level lexical overlap, not knowledge retention. Since Tables 1–3 and the abstract's headline figures are computed with this metric, the quantitative foundation of the central claim is not established. The human-in-the-loop validation in §4.1 filters generated questions, not model responses, so it does not correct this flaw.","section":"§5.2, Algorithm 2"},{"comment":"The abstract states that \"62.5% successfully retrieved forgotten Harry Potter facts from WHP-unlearned Llama\", but Table 3 reports 62.5% in the WHP Llama retain-set Direct row, which corresponds to unfair suppression of retained knowledge, not retrieval of forgotten facts. The forget-set Direct rate for WHP Llama is 27.5%. The headline number is therefore misattributed to the wrong quantity, and the actual forget-set retrieval rates are materially lower. This internal inconsistency must be resolved before any of the paper's quantitative conclusions can be interpreted.","section":"Abstract, Table 3"},{"comment":"The claim that \"RMU consistently exhibits a 100% failure rate across all question types\" is not supported by any table or per-question data in the paper. The single illustrative response about Platform 9 3/4 is ungrammatical repetition (\"a heart, a bird, a fish...\") that contains none of the Harry Potter keywords from Appendix A.1. Under the paper's own keyword-presence metric, this response would not be counted as a success. Either the 100% figure is based on a different, undescribed scoring rule, or it is unsupported. This claim is central to the paper's ranking of unlearning methods and needs a data table or a precise definition.","section":"§5.2 (RMU)"},{"comment":"The UNSTAR discussion in §5.2 attributes results to both Llama and Mistral, including \"0% failure for Llama and Mistral\" in the Indirect category and \"a maximum of 17.5% failure in the Forget Set (Llama, Implied prompts)\". However, Table 2 only reports results for Mistral; the 17.5% value appears in the Mistral Implied cell, not in any Llama row. The conclusion that UNSTAR is \"the most robust technique\" is therefore not backed by the displayed data for one of the two LLMs the text claims to evaluate.","section":"Table 2, §5.2 (UNSTAR)"},{"comment":"The attack construction has a circularity problem: probing questions are generated by decomposing the target answer with a support LLM that has full knowledge of the forget set (e.g., \"Where did Harry Potter study?\" is decomposed into steps that explicitly name Hogwarts), and the keyword list in Appendix A.1 (Step 5) is derived from the same entities and relationships. Success is then defined as the presence of those very keywords in the unlearned model's output. This procedure partially guarantees leakage by construction: any model that talks about Hogwarts, Voldemort, or Gryffindor—even to deny their existence or to hallucinate—will be scored as a success. To support the claim that step-by-step reasoning \"recalls erased content\", the evaluation must measure whether the response actually entails the target fact (e.g., human-annotated entailment labels or a follow-up question whose correct answer is not one of the seed keywords).","section":"§4.1, Appendix A.1"}],"minor_comments":[{"comment":"The Introduction says \"comprehensive evaluations of three state-of-the-art unlearning techniques (WHP, RMU, OPT-OUT, UNSTAR)\", but the parenthetical lists four methods; the count should be four.","section":"§1 (Introduction)"},{"comment":"The retention condition in Eq. (1) is written as \"M(q, ϕ′) ̸= a\" for retain-set questions, which is identical in form to the forgetting condition and contradicts the prose that the model \"should continue to provide correct answers\". The inequality should be an equality.","section":"§3 (Preliminaries, Eq. (1))"},{"comment":"There are several typographical inconsistencies in method names: \"OPT-O UT\" appears in §5.2, \"S LEEK\" appears in the Figure 1 caption and in Appendix A.2, and \"LLaMa\" and \"Llama\" are used interchangeably. These should be standardized.","section":"§5.2 and Figure 1 caption"},{"comment":"Step 4 says \"Cluster based on computed embeddings of responses and filter\", but Algorithm 1 (Step 2) clusters the generated questions, not the responses; the description should be corrected to match the algorithm.","section":"Appendix A.1, Step 4"},{"comment":"The GPT score prompt does not specify the GPT model version, decoding parameters, or any validation against human judgments (e.g., inter-annotator agreement). Since the GPT score is the only metric for Implied questions, this missing detail limits reproducibility.","section":"Appendix A.2"},{"comment":"All tables report single-point success rates without confidence intervals, standard deviations, or significance tests. Given that question generation and model decoding are stochastic, the differences between methods and conditions may not be reliable without variance estimates.","section":"Tables 1–3"}],"recommendation":"reject","confidential_remarks":"The paper addresses a timely and important problem, and the attack framework is clearly articulated. However, the evaluation methodology is invalid for measuring knowledge retention: the keyword-presence metric counts denials and hallucinations as leakage, the headline 62.5% is taken from a retain-set cell rather than the forget-set result, and the RMU and UNSTAR claims are not backed by the displayed data. These are load-bearing issues that cannot be fixed by local edits. A future revision with a validated knowledge-entailment metric and corrected reporting could be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nYou should know this paper before citing it: the core idea—using step-by-step reasoning to decompose a forgotten fact into subquestions—is a legitimate and potentially useful extension of existing adversarial attacks on unlearning. The qualitative examples in Section 5.2 do show that OPT-OUT and WHP leak related information through indirect prompts. That part is worth reading.\n\nBut the quantitative backbone is broken. The abstract claims 62.5% of adversarial prompts retrieved forgotten Harry Potter facts from WHP-unlearned Llama, but Table 3 shows the only 62.5% cell is the retain set, not the forget set. The same table shows retain-set failure rates of 51% total, while the text insists WHP 'completely avoids suppressing valid retain-set knowledge.' That is a direct contradiction. RMU is said to have a 100% failure rate, but no RMU table appears anywhere. UNSTAR results are shown only for Mistral, though the prose claims results for both Llama and Mistral. No code, data, or error bars are provided.\n\nThe deeper problem is the success metric. Algorithm 2 scores direct and indirect responses by counting keywords from a human-curated list. The paper's own WHP example shows the failure mode: asked 'What is the name of the school where Harry Potter was sorted?', the unlearned model responds that it has no information on a school called 'Hogwarts' and that the series was created by Michael Coleman. That response contains multiple keywords and would be scored as a successful extraction, even though it is a denial plus a hallucination. So the 62.5% figure and the Direct/Indirect rates in the tables are not evidence of knowledge recovery; they are partly counts of surface-level lexical overlap.\n\nWhat is genuinely new here is the systematic generation of decomposed probing questions, and the direct/indirect/implied categorization is a sensible way to think about attack surfaces. But the paper does not yet establish that these prompts recover knowledge rather than trigger keyword-friendly refusals. A cleaner protocol would use human or model-based verification of whether the target fact is actually present, and would correct the abstract/table mismatches.\n\nThis is the kind of paper a serious editor could send to review because the problem is real and the attack idea is testable, but it needs major revision before the numbers can be trusted. For now, treat it as a pointer to an attack strategy, not as a reliable evaluation of unlearning methods.\n\nRecommendation: engage with the idea, but don't take the reported success rates at face value.","headline":"The attack idea is a reasonable extension of existing jailbreaking work, but the paper's own tables contradict its abstract and the keyword metric counts denials as leakage, so the headline numbers don't hold.","tokens_in":13978,"tokens_out":4804,"would_cite":false,"duration_ms":47235,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Step-by-step prompts recover 'erased' knowledge from unlearned LLMs","keywords":["LLM unlearning","machine unlearning","adversarial attack","step-by-step reasoning","chain-of-thought prompting","knowledge leakage","Harry Potter unlearning benchmark","black-box attack"],"falsifier":"Have human raters judge whether each keyword-positive response in SLEEK actually entails the target fact, or check a control condition where the unlearned model is asked about a fact whose keywords appear in a wrong answer. If keyword-positive responses are mostly non-entailing or fabricated, the reported 62.5% retrieval rate would be an artifact of the scoring rule rather than a measure of knowledge retention.","tokens_in":12919,"feed_emoji":"🔓","tokens_out":6825,"duration_ms":71047,"temperature":0.7,"pith_summary":"The paper argues that current LLM unlearning methods mostly suppress knowledge rather than erase it, and that step-by-step reasoning prompts can pull that suppressed knowledge back into the open. The authors introduce SLEEK, a black-box attack that turns a single forget-set question into many probe questions by having a support LLM reason through the fact and generating a question for each intermediate step. Tested against four unlearning techniques and two models, the attack retrieves supposedly erased content in a large share of attempts, including 62.5% of adversarial prompts against WHP-unlearned Llama, and also exposes cases where unlearning suppresses unrelated retained knowledge. The paper's conclusion is that unlearning evaluations must count indirect and inferred answers as failures, not just direct answers to the original question.","feed_headline":"Step-by-step prompts recover 'erased' knowledge from unlearned LLMs","feed_subtitle":"An attack called SLEEK makes supposedly unlearned models reveal forgotten Harry Potter facts in 62.5% of tries.","key_machinery":"The central object is SLEEK, a black-box attack pipeline built on the idea of decomposing a target fact into reasoning steps and re-asking each step as its own question. It works by (i) prompting a support LLM to 'think step by step' about a forget-set question, (ii) extracting intermediate facts from that trace and generating a question per fact, (iii) deduplicating questions with agglomerative clustering on sentence-transformer embeddings, (iv) human validation and iterative expansion of the question set, and (v) probing the unlearned model and classifying each response as direct, indirect, implied, or irrelevant. The classification scheme plus keyword-presence and GPT scoring is what converts a response into a measured unlearning failure or a measured suppression of retained knowledge.","core_discovery":"SLEEK claims to demonstrate that 'erased' knowledge remains recoverable through step-by-step reasoning. The attack generates reasoning traces from a support LLM, decomposes them into individual knowledge points, and creates one question per point; the question set is then deduplicated by embedding clustering, filtered by human annotators, expanded iteratively, and used to probe the unlearned model. Responses are categorized as direct, indirect, implied, or irrelevant, with leakage detected through keyword presence and a GPT-based inference score for implied answers. The headline results are that 62.5% of adversarial prompts retrieved forgotten Harry Potter facts from WHP-unlearned Llama, 50% exposed unfair suppression of retained knowledge, and RMU failed every probe, while UNSTAR resisted best yet still leaked in some categories.","pith_inferences":["A natural extension would turn SLEEK into a standard red-team harness for unlearned models before deployment, since it needs no weights or gradients.","The keyword-presence metric could overstate leakage when the unlearned model generates keyword-rich but factually wrong text; human or entailment-based validation of responses would give a truer retention rate.","The same reasoning-decomposition trick might transfer to other input modalities, such as retrieving erased facts through translation, summarization, or multilingual paraphrases, though the paper does not test those.","If the mechanism is that unlearning never removes latent traces, defense may require corrupting or randomizing internal representations rather than filtering outputs; that is an inference, not a claim the paper makes."],"forward_implications":["Unlearning evaluations that only re-ask the original forget-set questions will miss most residual knowledge, because SLEEK recovers it through rephrased and inferred queries.","Deployers should treat indirect mentions and logical implications as leakage events, not as acceptable partial forgetting.","Refusal-based and hallucination-based unlearning methods (OPT-OUT and WHP in this study) can appear safe on direct prompts while still leaking through background knowledge.","Retain-set probing matters as much as forget-set probing: the attack found 50% suppression of retained knowledge in WHP-unlearned Llama, meaning unlearning can silently damage model utility.","An adversary with only API access plus a support LLM can run the full attack, so black-box exposure alone is not a safeguard."],"supporting_citations":[{"why":"Supplies WHP, the 'Who's Harry Potter' unlearning method whose Llama unlearned model SLEEK retrieves forgotten facts from.","marker":"Liu et al. (2024b)"},{"why":"Supplies OPT-OUT, the optimal-transport unlearning method that SLEEK shows leaks through indirect questions.","marker":"Ma et al. (2025)"},{"why":"Supplies RMU, the representation-control unlearning method that SLEEK finds 100% vulnerable.","marker":"Li et al. (2024)"},{"why":"Supplies UNSTAR, the counterfactual-based unlearning method that SLEEK finds most resistant.","marker":"Sinha et al. (2024)"},{"why":"Supplies the original Harry Potter unlearning benchmark and the notion of approximate unlearning that SLEEK evaluates.","marker":"Eldan & Russinovich"},{"why":"Supplies the all-MiniLM-L6-v2 sentence transformer used for semantic deduplication of generated questions.","marker":"Wang et al. (2020)"}],"fun_headline_variants":["SLEEK attack exposes 'erased' knowledge in unlearned LLMs","Step-by-step backdoor retrieves forgotten LLM knowledge","Reasoning prompts sneak past LLM unlearning","62.5% of prompts reveal 'erased' Harry Potter facts","LLM unlearning fails: reasoning recovers hidden knowledge"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack's success metric assumes that when the unlearned model's output contains a keyword such as 'Hogwarts' or 'Gryffindor', the model genuinely knows the forgotten fact, rather than merely echoing vocabulary without the underlying knowledge.","fun_headline_variants_meta":{"raw":{"variants":["SLEEK attack exposes 'erased' knowledge in unlearned LLMs","Step-by-step backdoor retrieves forgotten LLM knowledge","Reasoning prompts sneak past LLM unlearning","62.5% of prompts reveal 'erased' Harry Potter facts","LLM unlearning fails: reasoning recovers hidden knowledge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000238,"raw_usage":{"total_tokens":1540,"prompt_tokens":1003,"completion_tokens":537,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":619,"completion_tokens_details":{"reasoning_tokens":450}},"tokens_in":619,"tokens_out":537,"duration_ms":5755,"temperature":1.0,"reasoning_tokens":450,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:51:33.070588+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have human raters judge whether each keyword-positive response in SLEEK actually entails the target fact, or check a control condition where the unlearned model is asked about a fact whose keywords appear in a wrong answer. If keyword-positive responses are mostly non-entailing or fabricated, the reported 62.5% retrieval rate would be an artifact of the scoring rule rather than a measure of knowledge retention.","supporting_citations":[],"review_version":1}