{"id":"d6a38666-ea79-4223-8bbb-88bd4064ae1d","arxiv_id":"2606.25001","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Output forgetting in machine unlearning overestimates success because unlearned models exhibit structured representation mismatches relative to retraining from scratch.","lead":"This paper finds that machine unlearning methods succeeding at output forgetting still show structured mismatches in representation space compared to a model retrained from scratch without the forget data. A smart generalist might read it to see why current tests for whether AI has truly removed specific data may be insufficient for privacy or compliance needs.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.3","headline":"Retraining may not be the sole valid reference for correct forgetting; alternative representationally distinct forgetting states could still prevent data recovery.","rationale":"The reader's weakest_assumption directly identifies the load-bearing point. The empirical observation of mismatch is robust to this concern; only the interpretation that mismatch equals overestimation of success depends on the uniqueness of the retraining reference. No other internal inconsistency (e.g., in metrics or experimental design) appears more central given the abstract and the stated claim.","tokens_in":1700,"tokens_out":340,"duration_ms":18982,"concrete_test":"Train 5 independent retrained models from scratch (different seeds, same hyperparameters) on the retain set only; compute pairwise representation distances (e.g., centered kernel alignment or cosine distance on penultimate layer activations) among the retrained models versus between each unlearned model and the retrained ensemble. If mean inter-retrained distance exceeds mean unlearned-to-retrained distance on forget samples, the reference is not unique and the mismatch does not imply incomplete forgetting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that deviation from the retrained model's representations constitutes incomplete forgetting. The paper treats the retrained model as the operational gold standard for representation-space forgetting and interprets any structured mismatch (forget/retain asymmetry, directional concentration) as evidence that output forgetting overestimates success. This assumes retraining is the unique or canonical forgetting trajectory. If other models can achieve effective forgetting (no membership inference success, no data reconstruction) via different representational adjustments that do not match the retrained model, then the observed mismatch does not necessarily indicate failure of unlearning.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that standard output-level metrics for machine unlearning (e.g., forget-set accuracy, logit-based membership inference) systematically overestimate success. Using the retrained model (trained from scratch on retain data only) as an operational reference for correct representation-space forgetting, the authors present theoretical analysis and empirical results across multiple unlearning methods, datasets, and models. They report that unlearned models exhibit structured mismatches: partial alignment with retraining on forget samples, greater inconsistency on retain samples, and residual discrepancy concentrated along retraining-related directions rather than diffuse.","tokens_in":1811,"tokens_out":531,"duration_ms":16916,"significance":"If the empirical patterns hold, the work identifies a concrete limitation in current MU evaluation practices and motivates stronger, representation-consistent criteria. The multi-method, multi-dataset empirical component and the operational use of a retrained reference are strengths that make the overestimation claim testable and falsifiable.","major_comments":[{"comment":"Abstract and §3 (theoretical analysis): the central claim that output forgetting 'overestimates success' is load-bearing on the premise that deviation from the retrained model's representations constitutes incomplete forgetting. The manuscript does not provide a formal argument or empirical test showing that no other representationally distinct trajectories can achieve effective forgetting (zero membership inference, no reconstruction) while differing from the retrained model; this assumption requires explicit justification or a counter-example analysis.","section":"Abstract, §3"},{"comment":"Empirical results section (likely §4–5): the reported forget/retain asymmetry and directional concentration are interpreted as evidence of incomplete forgetting, but the paper does not report whether these mismatches correlate with actual downstream risks (e.g., increased reconstruction success or membership inference beyond output level). Without that link, the structured mismatch alone does not yet demonstrate overestimation of unlearning success.","section":"Empirical results (§4–5)"}],"minor_comments":[{"comment":"Notation for 'retraining-consistent representation forgetting' is introduced in the abstract but would benefit from an explicit definition or equation in the methods section to avoid ambiguity with standard representation similarity measures.","section":"Methods"},{"comment":"Figure captions should explicitly state the number of runs, random seeds, and statistical significance tests used for the reported asymmetries and directional concentrations.","section":"Figures"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the scope of a machine-learning venue focused on evaluation and privacy; no obvious citation-pattern issues."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. The comments highlight important areas for strengthening the justification of our central claim and the empirical linkage to downstream risks. We address each point below and will incorporate revisions accordingly.","responses":[{"response":"We agree that the operational use of the retrained model as the reference for true forgetting requires explicit justification. In the revised manuscript, we will expand §3 with a formal argument establishing that retraining from scratch on the retain set is the unique trajectory guaranteeing removal of forget-set influence (as any representationally distinct model retains latent structure from the forget data). We will also include a brief discussion of why output-level success on alternative trajectories does not constitute complete forgetting under a representation-consistent definition.","revision_made":"yes","referee_comment":"[Abstract, §3] Abstract and §3 (theoretical analysis): the central claim that output forgetting 'overestimates success' is load-bearing on the premise that deviation from the retrained model's representations constitutes incomplete forgetting. The manuscript does not provide a formal argument or empirical test showing that no other representationally distinct trajectories can achieve effective forgetting (zero membership inference, no reconstruction) while differing from the retrained model; this assumption requires explicit justification or a counter-example analysis."},{"response":"We acknowledge this limitation in the current empirical presentation. In the revised version of §4–5, we will add correlation analyses between the reported representation mismatch metrics (forget/retain asymmetry and directional concentration) and downstream risks, specifically reconstruction attack success rates and advanced membership inference performance beyond output logits. This will directly link the observed structured mismatches to overestimation of unlearning success.","revision_made":"yes","referee_comment":"[Empirical results (§4–5)] Empirical results section (likely §4–5): the reported forget/retain asymmetry and directional concentration are interpreted as evidence of incomplete forgetting, but the paper does not report whether these mismatches correlate with actual downstream risks (e.g., increased reconstruction success or membership inference beyond output level). Without that link, the structured mismatch alone does not yet demonstrate overestimation of unlearning success."}],"tokens_in":1384,"tokens_out":457,"duration_ms":18800,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core observation is that several unlearning methods produce models whose outputs look forgotten on standard checks, yet their internal representations still deviate from a retrained-from-scratch baseline in non-random ways: more aligned on forget samples than retain ones, with residuals concentrated along particular directions. That pattern is presented with some empirical breadth across methods and datasets, and it usefully flags that output-only metrics can miss internal structure.\n\nWhat the work does cleanly is make the comparison to retraining explicit and operational. Using the retrained model as a reference avoids some circularity that self-referential metrics can have, and the reported asymmetry plus directional concentration gives a concrete handle on where the mismatch lives.\n\nThe main limitation is the leap from \"does not match retraining\" to \"not true forgetting.\" The stress-test point holds: nothing in the abstract rules out the possibility that other representationally distinct states could still prevent membership inference or reconstruction. Treating retraining as the canonical target needs more defense—either theoretical or by showing that alternatives fail on downstream privacy tasks. Without that, the claim that output evaluation \"systematically overestimates\" success rests on one reference trajectory rather than a broader argument.\n\nThe math and data details are not visible here, so I cannot judge the derivations or error bars, but the framing itself is coherent and engages the literature on evaluation gaps. This is for researchers building or auditing unlearning benchmarks. A reader focused on privacy metrics or regulatory compliance would find the asymmetry results worth checking. It deserves peer review because the empirical pattern raises a real question about current practice, even if the interpretation of what counts as forgetting needs tightening.","headline":"The paper shows output unlearning can leave structured representation mismatches versus a retrained model, but treats that model as the sole valid reference without enough justification.","tokens_in":2286,"tokens_out":404,"would_cite":false,"duration_ms":12689,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Output forgetting in machine unlearning often leaves structured representation mismatches relative to retraining.","keywords":["machine unlearning","output forgetting","representation space","retraining","forget set","retain set","membership inference"],"falsifier":"An empirical result showing that representation distances and directional alignments between unlearned models and the retrained reference are statistically indistinguishable from random or zero across multiple methods would falsify the claim of systematic overestimation.","tokens_in":2594,"feed_emoji":"","tokens_out":615,"duration_ms":15961,"temperature":0.7,"pith_summary":"The paper questions whether low output accuracy or reduced membership inference on forget data truly certifies that a model has forgotten in the sense of matching a retrained model. It introduces retraining-consistent representation forgetting as a stronger check, comparing unlearned models to models trained from scratch without the forget data. Results across methods, datasets, and models show that output success frequently coexists with systematic residuals in representation space, including forget/retain asymmetry and directional concentration. A reader would care because this means current evaluations may certify apparent rather than actual forgetting, with implications for privacy and data removal claims. The work demonstrates that retraining exposes discrepancies hidden by output-only checks.","feed_headline":"Output forgetting overestimates machine unlearning success","feed_subtitle":"Unlearned models match retrained outputs but retain structured mismatches in representation space compared to models trained without the dat","key_machinery":"Retrain-consistent representation forgetting, which treats the model retrained from scratch without the forget data as the operational reference for correct forgetting in representation space.","core_discovery":"The central claim is that standard output-level evaluation can systematically overestimate unlearning success because output forgetting can coexist with retraining-inconsistent residuals in representation space. Under this lens, methods often partially align with retraining on forget samples, remain more inconsistent on retain samples, and leave residual discrepancy concentrated along retraining-related directions rather than diffuse.","pith_inferences":["Evaluation protocols may need to incorporate representation-space comparisons to the retrained reference to avoid overestimating forgetting.","Applications requiring verifiable data removal, such as regulatory compliance, could be affected if only output metrics are used.","Methods that directly optimize for reduced representation mismatch to retraining might address the identified gaps."],"forward_implications":["Unlearned models show partial alignment with retraining on forget samples but greater inconsistency on retain samples.","Residual mismatches concentrate along retraining-related directions rather than appearing diffuse in representation space.","Current methods often produce apparent output forgetting without achieving retraining-consistent forgetting.","Standard evaluations based on output accuracy or logit-level inference can overestimate true unlearning progress."],"fun_headline_variants":["Output forgetting masks representation mismatches in unlearning","Unlearning seems successful but leaves structured residuals","Standard output checks overestimate machine unlearning success","Representation space exposes unlearning overestimation","Unlearned models partially align but mismatch retraining"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The retrained model trained from scratch without the forget data serves as a valid operational reference for what correct forgetting should look like in representation space.","fun_headline_variants_meta":{"raw":{"variants":["Output forgetting masks representation mismatches in unlearning","Unlearning seems successful but leaves structured residuals","Standard output checks overestimate machine unlearning success","Representation space exposes unlearning overestimation","Unlearned models partially align but mismatch retraining"]},"model":"grok-4.3","cost_usd":0.003282,"raw_usage":{"total_tokens":1743,"prompt_tokens":645,"num_sources_used":0,"completion_tokens":64,"cost_in_usd_ticks":32824500,"prompt_tokens_details":{"text_tokens":645,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1034,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":645,"tokens_out":64,"duration_ms":6131,"temperature":1.0,"reasoning_tokens":1034,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T00:03:32.594864+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An empirical result showing that representation distances and directional alignments between unlearned models and the retrained reference are statistically indistinguishable from random or zero across multiple methods would falsify the claim of systematic overestimation.","supporting_citations":[],"review_version":1}