{"id":"ef8e834f-228c-423f-b457-ce619a35ee4d","arxiv_id":"2604.23854","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Standard unlearning raises false-negative rates and clinical risk in medical image models, but a modified SalUn-CRA variant using entropy-based forgetting for malignant samples matches full retraining risk while still removing data influence.","lead":"This paper tests whether common machine unlearning methods keep medical image classifiers safe by checking if they raise the rate of missed malignant cases. A smart generalist should read it because data privacy rules now require forgetting patient data, yet doing so carelessly could harm diagnosis accuracy in real healthcare systems.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Global Risk metric depends on unspecified asymmetric cost ratios whose clinical grounding is unverified","rationale":"The reader's weakest assumption directly identifies the metric's real-world fidelity; the concrete test above isolates whether that assumption is load-bearing by checking sensitivity of the headline comparison. No other internal inconsistency (e.g., in the unlearning definition or dataset splits) appears more decisive given the information supplied.","tokens_in":1723,"tokens_out":346,"duration_ms":17915,"concrete_test":"Recompute all reported Global Risk values on both DermaMNIST and PathMNIST using the exact cost ratio from the paper plus two additional ratios (half and double the original weight on FN); if the relative ordering between SalUn-CRA, standard SalUn, and retraining reverses for any ratio, the clinical-risk claim is not robust to plausible cost variation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline result—that standard unlearning raises clinical risk while SalUn-CRA matches retraining—rests on the Global Risk score, which weights false negatives more heavily than false positives. The abstract and method description give no explicit cost ratio, no derivation from clinical guidelines, and no sensitivity sweep. If the chosen ratio (whatever it is) is even moderately different from real diagnostic costs on these tasks, an observed rise in FN rate need not imply higher real-world risk, and the claimed superiority of entropy-based relabeling over random labeling could be an artifact of that weighting. The entropy mechanism itself is described only at high level (“entropy-based forgetting for malignant samples”), leaving open whether it systematically alters calibration or introduces new undetected failure modes not captured by the chosen metric.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that standard machine unlearning methods (Fine-Tuning, Random Labeling, SalUn) applied to binary medical image classifiers on DermaMNIST and PathMNIST can reduce test utility while increasing false-negative rates, thereby raising clinical risk when evaluated under asymmetric error costs. It proposes SalUn-CRA, a variant that substitutes entropy-based forgetting for random relabeling of malignant samples in the forget set, and reports that this variant achieves lower or comparable Global Risk scores to full retraining while retaining unlearning effectiveness at 20% and 50% data removal rates.","tokens_in":1882,"tokens_out":585,"duration_ms":64617,"significance":"If the empirical comparisons hold after supplying the missing metric definitions and implementation details, the work is significant because it shifts unlearning evaluation in medical AI from purely privacy- and utility-focused metrics toward clinically asymmetric risk. The introduction of an entropy-based mitigation and the explicit use of Global Risk provide a concrete example of how domain-specific safety considerations can be integrated into unlearning pipelines.","major_comments":[{"comment":"Abstract and Methods: The Global Risk metric with asymmetric costs is the sole basis for the headline claim that standard unlearning amplifies clinical risk and that SalUn-CRA matches retraining. No formula, explicit cost ratio (e.g., FN:FP weighting), derivation from clinical guidelines, or sensitivity analysis is supplied, rendering the directional results on false-negative increases unverifiable and potentially sensitive to the arbitrary choice of ratio.","section":null},{"comment":"Methods: The entropy-based forgetting mechanism for malignant forget-set samples in SalUn-CRA is described only at high level without pseudocode, exact entropy threshold or relabeling rule, or analysis of side effects on model calibration or new undetected failure modes. This mechanism is load-bearing for the proposed superiority over Random Labeling.","section":null},{"comment":"Experiments: No details are given on statistical significance testing, hyperparameter selection, exact implementation of the entropy mechanism or baselines, or the precise composition of the forget sets at 20% and 50% removal. These omissions prevent assessment of whether the reported risk reductions are robust or reproducible.","section":null}],"minor_comments":[{"comment":"Abstract: The phrase 'preserving unlearning effectiveness' should be accompanied by the concrete privacy or membership-inference metrics used to support it.","section":null},{"comment":"The manuscript would benefit from an explicit limitations paragraph discussing how well the chosen datasets and binary classification setting generalize to real multi-class clinical workflows.","section":null}],"recommendation":"major_revision","confidential_remarks":"The central empirical claims rest on an ad-hoc metric whose clinical grounding is not demonstrated; this is a scope and soundness concern for a medical-AI venue. Verify that the authors have disclosed all prior related work on risk-aware unlearning."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive and detailed review. We address each major comment below and will revise the manuscript to incorporate the requested clarifications and details.","responses":[{"response":"We agree that the Global Risk metric requires fuller specification to support the claims. In the revised manuscript we will add the explicit formula, the chosen asymmetric cost ratio (with justification drawn from clinical literature on the higher cost of missed malignancies), its derivation, and a sensitivity analysis across plausible ratios. These additions will make the reported directional effects on false-negative rates verifiable and allow readers to assess robustness.","revision_made":"yes","referee_comment":"[—] Abstract and Methods: The Global Risk metric with asymmetric costs is the sole basis for the headline claim that standard unlearning amplifies clinical risk and that SalUn-CRA matches retraining. No formula, explicit cost ratio (e.g., FN:FP weighting), derivation from clinical guidelines, or sensitivity analysis is supplied, rendering the directional results on false-negative increases unverifiable and potentially sensitive to the arbitrary choice of ratio."},{"response":"We acknowledge that the entropy-based forgetting component is currently described at too high a level. The revision will include pseudocode for the full procedure, the precise entropy threshold and relabeling rule applied to malignant samples, and an analysis of side effects on calibration and potential new failure modes. This will strengthen the justification for SalUn-CRA relative to standard random labeling.","revision_made":"yes","referee_comment":"[—] Methods: The entropy-based forgetting mechanism for malignant forget-set samples in SalUn-CRA is described only at high level without pseudocode, exact entropy threshold or relabeling rule, or analysis of side effects on model calibration or new undetected failure modes. This mechanism is load-bearing for the proposed superiority over Random Labeling."},{"response":"We will expand the Experiments section and supplementary material with the missing information: the statistical tests employed (including p-value thresholds), the hyperparameter selection protocol, exact implementation details for the entropy mechanism and all baselines, and the precise composition of the forget sets (including how malignant samples were sampled at each removal rate). These additions will support reproducibility and allow evaluation of result robustness.","revision_made":"yes","referee_comment":"[—] Experiments: No details are given on statistical significance testing, hyperparameter selection, exact implementation of the entropy mechanism or baselines, or the precise composition of the forget sets at 20% and 50% removal. These omissions prevent assessment of whether the reported risk reductions are robust or reproducible."}],"tokens_in":1456,"tokens_out":558,"duration_ms":41112,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that common unlearning methods (fine-tuning, random labeling, SalUn) raise false-negative rates on DermaMNIST and PathMNIST when 20-50% of data is removed, while the authors' SalUn-CRA tweak using entropy-based forgetting on malignant samples brings the risk back in line with full retraining. That directional finding on clinical safety is the concrete addition here. It takes existing unlearning techniques and tests them where error costs are asymmetric, which most prior work skips. The evaluation on two standard medical datasets and the explicit comparison to retraining is straightforward and useful for anyone thinking about deployment. The paper does a reasonable job flagging that privacy fixes can create new safety problems in diagnosis tasks. The soft spot is the Global Risk metric itself. The abstract and summary give no explicit cost ratio between false negatives and false positives, no derivation from clinical guidelines, and no sensitivity check. If that weighting is arbitrary or off by even a moderate amount, the claim that SalUn-CRA matches retraining could shift. The entropy mechanism is described at a high level without details on how it avoids new calibration issues or undetected failure modes. No statistical tests or hyperparameter reporting appear in the high-level results either. This is for people working on unlearning or medical AI safety who want to see the risk angle spelled out. A reader already familiar with the datasets will get the most out of it. It deserves peer review because the question is real and the setup is accessible, even though the current evidence is preliminary and the metric needs clearer grounding before the conclusions can be taken as solid.","headline":"Standard unlearning increases false negatives on these medical datasets, but the Global Risk claims rest on an unspecified weighting that isn't clinically justified in the text.","tokens_in":2346,"tokens_out":397,"would_cite":false,"duration_ms":24132,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Standard unlearning methods can raise false-negative rates in medical image classification, increasing clinical risk, while a modified SalUn-CRA variant keeps risk comparable to full retraining.","keywords":["machine unlearning","clinical risk","medical image classification","false negatives","data removal","DermaMNIST","PathMNIST","deep learning safety"],"falsifier":"A clinical deployment on new patient images where SalUn-CRA produces a higher false-negative rate for confirmed malignant cases than a model retrained from scratch on the remaining data.","tokens_in":2620,"feed_emoji":"🩺","tokens_out":765,"duration_ms":45279,"temperature":0.7,"pith_summary":"The paper tests whether machine unlearning, which removes selected training examples from a deployed model, preserves safety when the task is binary medical image classification. Common techniques such as fine-tuning, random labeling, and SalUn lower overall accuracy and raise the rate of missed malignant cases, which carries asymmetric harm in diagnosis. The authors introduce SalUn-CRA, which substitutes entropy-based forgetting for malignant samples in the forget set instead of random relabeling. On DermaMNIST and PathMNIST at 20 % and 50 % removal rates, SalUn-CRA produces Global Risk scores with asymmetric costs that match or beat those of full retraining while still achieving the requested forgetting. The work concludes that clinical risk metrics must be part of unlearning evaluation in medical systems rather than relying on utility or privacy metrics alone.","feed_headline":"Unlearning medical images can raise missed-diagnosis risk","feed_subtitle":"A risk-aware variant of SalUn matches full retraining safety by using entropy-based forgetting for malignant cases instead of random labels.","key_machinery":"SalUn-CRA, a variant of SalUn that replaces random relabeling with entropy-based forgetting targeted at malignant samples in the forget set, combined with Global Risk metrics that apply asymmetric costs to false negatives versus false positives.","core_discovery":"Standard unlearning strategies (Fine-Tuning, Random Labeling, and SalUn) may reduce test utility while increasing false-negative rates, thereby amplifying clinical risk. SalUn-CRA, which replaces random relabeling with entropy-based forgetting for malignant samples, achieves lower or comparable clinical risk to full retraining while preserving unlearning effectiveness on DermaMNIST and PathMNIST under 20 % and 50 % data removal.","pith_inferences":["The same risk-aware forgetting strategy could be tested on multi-class or segmentation tasks common in radiology.","Regulatory frameworks for medical AI data deletion may need to require explicit clinical-risk audits rather than only membership-inference or accuracy checks.","If entropy-based forgetting generalizes, it could reduce the computational cost of safe unlearning by avoiding full retraining in regulated environments."],"forward_implications":["Unlearning without clinical-risk awareness can increase the probability of missed diagnoses even when overall accuracy appears acceptable.","Replacing random relabeling with entropy-based forgetting for malignant samples prevents the model from forming harmful benign associations.","Global Risk with asymmetric costs provides a more appropriate validation signal than standard accuracy or privacy metrics for medical unlearning.","SalUn-CRA matches or undercuts the clinical risk of full retraining at both 20 % and 50 % removal rates on the tested dermatology and pathology datasets."],"fun_headline_variants":["Unlearning medical images raises missed-diagnosis risk","Standard unlearning increases false negatives in diagnostics","SalUn-CRA achieves lower clinical risk than standard methods","Risk-aware unlearning matches retraining safety in medical AI"],"cache_read_input_tokens":64,"weakest_assumption_plain":"That Global Risk metrics with asymmetric costs computed on DermaMNIST and PathMNIST accurately represent real-world clinical safety trade-offs and that entropy-based forgetting introduces no new undetected risks.","fun_headline_variants_meta":{"raw":{"variants":["Unlearning medical images raises missed-diagnosis risk","Standard unlearning increases false negatives in diagnostics","SalUn-CRA achieves lower clinical risk than standard methods","Risk-aware unlearning matches retraining safety in medical AI"]},"model":"grok-4.3","cost_usd":0.005814,"raw_usage":{"total_tokens":2685,"prompt_tokens":665,"num_sources_used":0,"completion_tokens":60,"cost_in_usd_ticks":58140500,"prompt_tokens_details":{"text_tokens":665,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1960,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":665,"tokens_out":60,"duration_ms":26536,"temperature":1.0,"reasoning_tokens":1960,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-08T05:54:26.365297+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A clinical deployment on new patient images where SalUn-CRA produces a higher false-negative rate for confirmed malignant cases than a model retrained from scratch on the remaining data.","supporting_citations":[],"review_version":1}