{"id":"4d1ae3fe-0a4a-433f-8335-54d8cdc81bc9","arxiv_id":"2502.10288","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"MixUnlearn trains a generator to create adversarial mixup samples from forgetting and remaining data, then uses contrastive objectives to forget the target while keeping other knowledge.","lead":"This paper proposes MixUnlearn, a machine unlearning method in which a small generator creates adversarial mixup images from data to be forgotten and data to be kept, and a contrastive loss trains the unlearning model on those hard examples. The authors report improved fidelity to a retrained model across several image benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed mechanism for mixup—that mixing Forgetting and Remaining samples reproduces catastrophic overlap—is not isolated in the ablations; replacing cross-set mixing with remaining-only mixing could give identical gains.","rationale":"The reader's weakest_assumption identifies the same load-bearing premise: that mixup samples formed from Forgetting and Remaining data mimic the conflict region, transferring to unseen remaining data. My reading of the manuscript confirms this is asserted without a theoretical derivation or a direct empirical isolation of the cross-set pairing. The ablations in Section 5.5 only vary the mixing rule (learnable vs. vanilla, different α) while keeping the cross-set structure, so they cannot distinguish the proposed mechanism from generic mixup regularization. This is precisely the sort of hidden-assumption risk that warrants a conditional verdict. The label-agnostic Sharpen target is a secondary fragility; the paper's own ablation (w/o Sharpen) shows only modest gains, and the label-agnostic variant consistently trails the label-aware one, so it does not threaten the main claim as directly. My proposed concrete test—remaining-only mixing—would settle whether the cross-set pairing is actually necessary. Since the reader already conditioned the verdict on this premise and my concern does not move the verdict to a different category, I recommend UNCHANGED (i.e., maintain CONDITIONAL).","tokens_in":23183,"tokens_out":8607,"duration_ms":87996,"concrete_test":"Run an ablation identical to the full MixUnlearn pipeline on CIFAR-10 and SVHN class-level unlearning, but replace the generator's input pair (xi ∈ Df, xj ∈ Dr) with two Remaining samples (xi, xj ∈ Dr), keeping all losses, hyperparameters, and training schedules the same. If Testr, Testf, and ASR relative to Retrain stay within 1% of the full MixUnlearn results (Table 1, 'Ours'), then cross-set mixing is not the source of the improvement and the central mechanism is unsupported; if performance degrades substantially, the Df×Dr pairing is necessary and the premise is provisionally validated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central contribution is the claim that mixup samples formed from one Forgetting sample (xi ∈ Df) and one Remaining sample (xj ∈ Dr) simulate the conflict region where forgetting harms retention, so regularizing the unlearner on these samples (Eq. 5) overcomes catastrophic unlearning. This premise is asserted in Section 4 and motivated only by the informal toy example in Figure 1. The ablation study (Section 5.5, Table 3) compares the full method against vanilla mixup (w/o MB) but always mixes Df with Dr; it never tests whether cross-set pairing is necessary. If any form of mixup regularization—e.g., mixing two Remaining samples—yields the same Testr/Trainr improvement over LAF, then the observed gains are not evidence for the proposed mechanism. Instead they would reflect generic smoothing from interpolation, and the adversarial generator's specifically cross-set objective would be unjustified. This is load-bearing because the method's novelty and the entire 'catastrophic effects' narrative depend on Df×Dr mixing being special.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MixUnlearn, a generator-unlearner framework for approximate machine unlearning. A MixBlock generator creates hard mixup examples by mixing one Forgetting sample with one Remaining sample, and an adversarial contrastive objective trains the generator to produce samples that make the unlearner reveal Forgetting information while losing Remaining knowledge. The unlearner is then regularized by two contrastive losses, one on the mixed samples (Eq. 5) and one on real data (Eq. 6), to forget the target data while retaining utility. Experiments are reported on CIFAR-10, SVHN, MNIST, and FASHION-MNIST, with additional ImageNet/ViT results, in both label-aware and label-agnostic settings, and compared against a wide range of baselines including Retrain, LAF, LAF+R, GLI, and a proposed L-Mix baseline. The paper claims that MixUnlearn significantly outperforms existing state-of-the-art unlearning techniques and overcomes catastrophic unlearning.","tokens_in":23383,"tokens_out":5474,"duration_ms":54568,"significance":"If the claimed effect is real, MixUnlearn is a practically useful approximate unlearning method: it preserves retention utility while erasing targeted data, works without labels, and is relatively cheap due to the lightweight 66K-parameter MixBlock. The paper has notable strengths: code is released; experiments use five seeds and report error bars; the ablation study covers several components; hyperparameter sensitivity is analyzed; robustness checks on noisy-label and semi-supervised settings are included; and the ImageNet/ViT extension broadens the evidence base. However, the central mechanism of the method, that cross-set Df-by-Dr mixup specifically simulates catastrophic overlap, is not isolated by the ablations, and the 'significant' improvement claim is not uniformly supported by the data-level results. These issues are fixable but require additional experiments and more careful claims.","major_comments":[{"comment":"The ablation study does not isolate the claimed mechanism that mixing Forgetting with Remaining samples is what generates the benefit. The 'w/o MB' ablation replaces the learned MixBlock with vanilla mixup but still mixes Df with Dr samples; it never tests a Remaining-only mixup (or a random-pair mixup) under the same losses Lmix plus Lreal. Consequently, the observed gains over LAF could be due to generic interpolation smoothing of decision boundaries rather than to the paper's proposed 'catastrophic overlap' mechanism. This is load-bearing because the adversarial generator's cross-set objective (Eq. 3) and the central narrative in Sections 1 and 4 depend on Df-by-Dr pairing being special. I ask the authors to add an ablation that mixes two Remaining samples (and, ideally, a Remaining-with-random-class mixing) using the same Lmix and Lreal losses, and to report Testr, Testf, Trainr, and Trainf for those variants.","section":"Section 5.5, Tables 3 and 4"},{"comment":"The abstract and Section 5.4 claim that MixUnlearn 'significantly outperforms existing state-of-the-art unlearning techniques' and 'achieves significantly greater gains' than L-Mix, but the data-level (Basic) label-agnostic results do not support the word 'significantly' in that setting. For example, on CIFAR-10, Ours has Test 84.82±1.39 versus L-Mix 84.56±1.46, Trainr 79.18±0.98 versus 79.01±1.78, and Trainf 78.48±1.25 versus 79.65±2.21; all differences are within one standard deviation, and similar overlap appears for SVHN and MNIST. The manuscript reports no significance tests (e.g., paired tests across the five seeds). Either provide such tests and restrict the 'significant' claim to the configurations where it holds, or temper the claim in the abstract and Section 5.4.","section":"Section 5.4, Table 2"},{"comment":"The hyperparameter selection protocol is not specified. Appendix A.1 lists large search grids for alpha, tau_gen, tau_mix, tau_real, omega, and the sharpen temperature T, and Appendix A.3 states that baseline-specific parameters are tuned 'to optimize performance,' but the manuscript does not state whether this tuning is performed on a validation split or on the same test metrics reported in Tables 1 and 2. If the reported test metrics are used to select hyperparameters, the comparisons are overoptimistic relative to a fair deployment scenario. Please specify the selection criterion and, if possible, show that the ranking of methods is stable under alternative reasonable hyperparameter choices.","section":"Section A.1 and Section 5.1"}],"minor_comments":[{"comment":"The caption contains a typo: 'sythesize' should be 'synthesize'.","section":"Figure 1 caption"},{"comment":"The module name is typeset as 'M ixBlock' in Section 4.1 and elsewhere; this should be 'MixBlock' consistently.","section":"Throughout"},{"comment":"The definition of SimLoss as (1 - cosine similarity) appears only after Eq. 3, which makes the sign of the adversarial objective hard to follow; please state the definition before Eq. 3 and briefly explain the sign convention in the text.","section":"Equation 3 and Section 4.1"},{"comment":"The sentence 'we train two 18-layer ResNet models on the CIFAR datasets' should say 'on CIFAR-10 and SVHN' for precision, since only CIFAR-10 is used.","section":"Appendix A.2"},{"comment":"The caption and the surrounding text repeat the same sentence about the time cost comparison on an NVIDIA GeForce RTX 3090; please remove the duplication.","section":"Appendix A.8, Figure 7 caption"},{"comment":"The caption says 'lambda is set as 0.5,' but the paper elsewhere samples lambda from a Beta distribution; please clarify whether this is a fixed value for visualization only.","section":"Figure 8 caption"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is well organized and the empirical study is fairly extensive, but the central mechanism claim needs a targeted ablation and the 'significant' language needs to be brought in line with the error bars. I believe these are fixable within the scope of the paper, so I recommend major revision rather than rejection. One additional concern for the editor: the paper does not report a validation protocol for hyperparameter tuning, which could affect the credibility of the comparisons against baselines whose parameters were also tuned."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the quick read: this is a workmanlike empirical paper that will be a useful data point in approximate unlearning, but the story it tells about why it works is not supported by its own ablations. The class-level gains are real; the data-level gains over the simple L-Mix baseline are mostly within error bars, so the 'significantly outperforms' claim in the abstract is too strong for the whole paper.\n\nWhat's genuinely new: combining a learnable adversarial mixup generator (borrowed from AdAutoMix) with contrastive unlearning losses that push the unlearner to forget the forgetting sample and retain the remaining sample is, as far as I know, novel. The framework handles both label-aware and label-agnostic settings, which is practically useful. The empirical footprint is solid: four datasets plus an ImageNet/ViT extension, five seeds, released code, and ablations showing that both Lreal and Lmix matter. The authors also honestly flag the open question of when to stop unlearning. Good.\n\nThe soft spots, in rough order of severity. First, the mechanism claim. The paper says mixup samples from Df x Dr simulate the catastrophic overlap region and that this is why regularization works. But the ablations always mix a forgetting sample with a remaining sample, both in the full method and in the 'w/o MB' vanilla mixup versions. Nobody tests whether mixing two remaining samples (or random pairs) gives the same boost. If it does, the observed gains are generic smoothing and the adversarial generator's cross-set objective has no special role. I agree with the stress-test note: this is a real gap in the evidence. It doesn't kill the empirical contribution, but it means the 'simulate the data susceptible to catastrophic effects' narrative is an assertion, not a demonstrated fact.\n\nSecond, hyperparameter selection. They search alphas, three temperatures, a sharpen temperature, omega, and unlearning epochs over wide grids, with no described validation procedure. On small datasets, picking the best from a big grid can account for exactly the kind of gains they report. This is a moderate concern that should have been addressed.\n\nThird, the significance claim is inflated for data-level unlearning. On CIFAR-10, SVHN, MNIST, and Fashion-MNIST, the data-level deltas over L-Mix are on the order of 0.1-0.2 points, well within standard deviations. The class-level gains are larger (e.g., +4 points on CIFAR-10 agnostic), so the method has real value there, but the paper should have reported the data-level results as comparable rather than part of a blanket superiority claim.\n\nWho is this for? Researchers working on approximate unlearning, especially those looking for empirical baselines in the mixup-for-unlearning space. It deserves a serious referee: it is a careful, reproducible empirical study with a testable central claim, even though that claim remains untested. I would not cite it as evidence for the catastrophic-overlap mechanism, but I would cite it as an empirical unlearning method with competitive class-level performance.","headline":"Solid empirical unlearning paper with a real class-level contribution, but the mechanism story is under-tested and the data-level gains over simple mixup are mostly within error bars.","tokens_in":23916,"tokens_out":3955,"would_cite":true,"duration_ms":38293,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MixUnlearn makes unlearning mimic retraining by regularizing on adversarial mixup samples.","keywords":["machine unlearning","catastrophic unlearning","mixup augmentation","adversarial training","contrastive loss","label-agnostic unlearning","data privacy"],"falsifier":"Train a classifier on two well-separated classes, run MixUnlearn to delete one class completely, and then measure test accuracy on the remaining class and membership-inference attack success rate against a model retrained without the deleted class. If the unlearned model drifts from Retrain on these metrics even though the mixup samples cover the interpolation zone, the claim that interpolated samples capture the catastrophic region is false. A sharper test is to introduce remaining-data points that are far from any convex combination of forgotten and remaining training samples and check whether the adversarial mixer can still generate the hard cases that cause catastrophic loss.","tokens_in":22971,"feed_emoji":"🧹","tokens_out":7051,"duration_ms":64815,"temperature":0.7,"pith_summary":"This paper is trying to establish that catastrophic unlearning—the failure mode where deleting one piece of data also damages knowledge the model should keep—can be overcome by regularizing the unlearner with synthesized mixup samples that inhabit the region where forgetting and retention interfere. It proposes MixUnlearn, a generator-unlearner framework in which a lightweight adversarial mixer creates hard interpolations between data to be forgotten and data to be retained, and contrastive losses then teach the unlearner to remove only the target information while preserving behavior on remaining and unseen data. The intended payoff is a practical approximate-unlearning method: a model that behaves like one retrained without the forgotten data, without the cost of full retraining, in both label-aware and label-agnostic settings. A sympathetic reading of the experiments is that MixUnlearn consistently lands closer to Retrain than existing unlearning baselines across four benchmark datasets and an ImageNet/ViT setup.","feed_headline":"Adversarial mixup samples tame catastrophic machine unlearning","feed_subtitle":"A generator-unlearner pair trains on hard synthetic mixes so deleted data is erased without dragging down retained knowledge.","key_machinery":"The load-bearing object is the generator-unlearner loop. A MixBlock module—a learnable attention-based mixer with roughly 66K parameters—takes one forgetting sample and one remaining sample and produces a mixed sample $x^{\\mathrm{mix}}_{ij} = g(x_i, x_j, \\lambda)$ with $\\lambda$ sampled from a Beta distribution. An adversarial contrastive loss trains this generator to produce hard samples that push the unlearner to reveal the forgetting sample's target while losing knowledge from the remaining sample. The unlearner is then optimized with two contrastive losses: $L_{\\mathrm{mix}}$ applies the reverse objective on the synthetic mixed samples, and $L_{\\mathrm{real}}$ reinforces forgetting and retention on the original samples, combined as $L_{\\mathrm{unlearn}} = L_{\\mathrm{mix}} + \\omega L_{\\mathrm{real}}$. In the label-agnostic variant, unavailable labels are replaced by sharpened predictions of the initial model, which concentrates the pseudo-target distribution. This mechanism is designed so that the unlearner sees exactly the interpolation zone where the paper argues catastrophic interference lives.","core_discovery":"On the paper's own terms, the central claim is that MixUnlearn outperforms existing state-of-the-art unlearning techniques in both label-agnostic and label-aware setups, and that its adversarial mixup regularization overcomes catastrophic effects so the unlearned model approximates a model retrained without the forgotten data. Concretely, the paper reports that its label-aware variant reaches remaining-class test accuracies close to Retrain on CIFAR-10 and SVHN—for example 87.10% and 93.95% in class-level unlearning—while driving the forgotten-class accuracy to roughly zero, and that its label-agnostic variant handles semi-supervised and noisy-label conditions without explicit labels. It also claims that the method is efficient: the learnable mixer has about 66,000 parameters and is updated only every few iterations, so the whole procedure is faster than teacher-student baselines and far cheaper than retraining.","pith_inferences":["An implication the paper leaves implicit is that unlearning difficulty is governed by feature-space geometry: MixUnlearn should need fewer mixup samples when classes are well separated and more when decision regions heavily overlap, a prediction that could be tested by varying class similarity.","The same generator-unlearner pattern could be carried to other modalities by defining the mixing function in an embedding space rather than raw input space—for example interpolating text or tabular representations—provided the interpolation remains semantically meaningful.","The paper leaves open whether samples outside the interpolation region, such as adversarial perturbations of remaining data, can also trigger catastrophic loss; if they can, a future extension would need to generate those cases as well.","Because the ablations show that the real-sample loss is essential, one actionable extension is to anneal its weight during unlearning, starting with strong retention and gradually increasing the mixup regularization."],"forward_implications":["Approximate unlearning can approach Retrain behavior in class-level and data-level deletion tasks, including membership-inference attack success rates close to retraining, on CIFAR-10, SVHN, MNIST, and Fashion-MNIST.","Deletion can be made label-agnostic: no labels for forgotten or remaining data are needed, which extends unlearning to semi-supervised and noisy-label settings.","A small adversarial mixer with about 66K parameters is enough to produce the hard examples, and because it is updated only every few iterations, the overhead stays below teacher-student unlearning baselines.","The learnable generator adds a real gain over vanilla mixup in this setting: replacing the generator with fixed-ratio interpolation degrades both remaining-class utility and forgetting completeness in the reported ablations.","The method transfers beyond small images: the reported ImageNet/ViT experiments show the same pattern of high remaining-class accuracy with near-total forgetting of targeted classes."],"supporting_citations":[{"why":"Supplies the mixup interpolation operation that MixUnlearn builds on.","marker":"(Zhang et al., 2018)"},{"why":"Provides the MixBlock module used as the learnable mixing function.","marker":"(Qin et al., 2024)"},{"why":"Supplies the Sharpen operation that generates pseudo-targets in the label-agnostic variant.","marker":"(Goodfellow et al., 2016)"},{"why":"Defines the LAF baseline and the label-agnostic unlearning setups and metrics the experiments adopt.","marker":"(Shen et al., 2024)"},{"why":"Introduces machine unlearning and the SISA exact-unlearning baseline that MixUnlearn is compared against.","marker":"(Bourtoule et al., 2021)"},{"why":"Provides the Boundary unlearning baseline that MixUnlearn must beat.","marker":"(Chen et al., 2023)"},{"why":"Provides the teacher-student T-S baseline.","marker":"(Chundawat et al., 2023a)"},{"why":"Provides the SCRUB baseline for comparison.","marker":"(Kurmanji et al., 2024)"},{"why":"Provides the GLI data-augmentation baseline and the KDE evaluation protocol for catastrophic utility drop.","marker":"(Choi et al., 2024)"},{"why":"Provides DSMixup, the mixup-based exact-unlearning method whose different goal the paper contrasts with its own.","marker":"(Zhou et al., 2022)"}],"fun_headline_variants":["Adversarial mixes erase data without wrecking model","Generator-unlearner pair beats catastrophic unlearning","MixUnlearn: tricky mixups keep unlearning precise","Hard synthetic mixes stop unlearning from forgetting too much"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that mixup samples formed by interpolating between forgotten and remaining data actually mimic the points where forgetting and retention conflict, so that regularizing the unlearner on these synthetic samples transfers to unseen remaining data.","fun_headline_variants_meta":{"raw":{"variants":["Adversarial mixes erase data without wrecking model","Generator-unlearner pair beats catastrophic unlearning","MixUnlearn: tricky mixups keep unlearning precise","Hard synthetic mixes stop unlearning from forgetting too much"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000188,"raw_usage":{"total_tokens":1344,"prompt_tokens":968,"completion_tokens":376,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":313}},"tokens_in":584,"tokens_out":376,"duration_ms":4246,"temperature":1.0,"reasoning_tokens":313,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T18:40:17.804220+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a classifier on two well-separated classes, run MixUnlearn to delete one class completely, and then measure test accuracy on the remaining class and membership-inference attack success rate against a model retrained without the deleted class. If the unlearned model drifts from Retrain on these metrics even though the mixup samples cover the interpolation zone, the claim that interpolated samples capture the catastrophic region is false. A sharper test is to introduce remaining-data points that are far from any convex combination of forgotten and remaining training samples and check whether the adversarial mixer can still generate the hard cases that cause catastrophic loss.","supporting_citations":[],"review_version":1}