REVIEW 4 major objections 5 minor 29 references
Bias-Aware Machine Unlearning: Towards Fairer Vision Models via Controllable Forgetting
T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read Post-hoc machine unlearning can cut vision-model subgroup disparities by up to 97%, the paper argues, with only minimal accuracy loss.
desk verdict Useful benchmark idea undermined by an undefined metric and an evaluation that shares the same subgroup axis with the forget set, so the headline fairness gains may be mechanical. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the bias-aware unlearning formulation: identify a biased subset $D_b$ of the training data, then update parameters so the model approximates one retrained on $D_{\text{train}} \setminus D_b$. The update mechanisms are gradient ascent on the forget-set loss with retain-set regularization, LoRA low-rank adapters maximizing forget loss while minimizing retain loss, a teacher-student distillation objective that aligns on retain data and diverges on forget data, and a counterfactual influence-function update. The evaluation machinery is Co-BUM, a weighted harmonic mean that normalizes utility, fairness (Demographic Parity and Equalized Odds), unlearning quality, membership-inference privacy, a
What would settle it
Take the unlearned CelebA and CUB models and evaluate them on a held-out test set where the spurious attribute is decorrelated from the label but sampled independently of the forget-set partition (e.g., newly collected bird poses or a gender-balanced smiling test set); if the demographic-parity and equalized-odds improvements disappear or accuracy collapses, the central claim that spurious correlations were removed would be refuted.
Extended reading notes
Core claim
The central claim is that selectively forgetting a hand-chosen biased subgroup, rather than retraining, can remove spurious correlations and reduce group disparities in vision classifiers. The paper formalizes bias-aware unlearning as updating model parameters so the resulting function approximates a model retrained on the data minus the biased subset, and tests gradient ascent, LoRA fine-tuning, teacher-student distillation, counterfactual debiasing, and exact retraining. The result pattern is context-dependent: boundary-pushing gradient ascent works best for diffuse pose bias, low-rank adapters work best for a localized patch artifact, and aggressive forgetting nearly eliminates demographi
Load-bearing premise
The validity of the headline gains rests on the assumption that unlearning on the hand-chosen biased subset—distant-pose birds, patched birds, male-smiling faces—is the right intervention axis, and that measuring fairness on exactly those subgroups reflects genuine debiasing rather than a mechanical prediction shift.
Editorial extensions
If this is right
- Deployed vision models that have learned spurious correlations can be corrected in minutes-to-hours on a single GPU rather than retrained from scratch.
- The best unlearning method depends on bias type: boundary-pushing methods for distributed pose bias, small adapter updates for localized shortcuts, and aggressive forgetting for entrenched attribute correlations.
- Fairness gains can co-occur with privacy gains: several methods lower membership-inference attack success as they lower demographic parity gaps.
- Demographic parity and equalized odds move apart after unlearning, so reporting only one fairness metric can overstate progress.
- Co-BUM provides a single number for comparing unlearning strategies across utility, fairness, quality, and privacy, aiding post-deployment correction choices.
Reading between the lines
- Editorial extension: because the forget set and the fairness evaluation set are the same hand-defined subgroup, the reported DP gains may partly reflect the mechanical effect of changing predictions on exactly those examples; testing on an independent spurious-correlation partition would separate genuine debiasing from memorization of the intervention.
- Editorial extension: the promise of avoiding full retraining would be strengthened by comparing against standard debiasing baselines such as reweighting, adversarial training, and data augmentation under matched compute budgets; the paper does not include that comparison.
- Editorial extension: a natural extension is sequential or joint unlearning of multiple correlated biases, since the paper treats each bias in isolation.
- Editorial extension: the Co-BUM normalization against a 'gold' retrained model requires access to a retrained reference model, which may limit its use exactly in the no-retraining deployment scenario the paper motivates.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes 'bias-aware machine unlearning' as a post-hoc debiasing tool for vision models. It evaluates Gradient Ascent, LoRA, SCRUB-style teacher-student distillation, Fast Model Debiasing (FMD), and hard unlearning on three datasets: CUB-200-2011 (pose bias), CIFAR-10 (synthetic patch bias), and CelebA (gender-smile correlation). The authors report large demographic-parity (DP) improvements, claim these are achieved with 'minimal accuracy loss,' and introduce a composite metric, Co-BUM, to jointly score utility, fairness, quality, privacy, and efficiency. The central claim is that unlearning can selectively remove spurious correlations and improve fairness without retraining, making it a practical alternative to conventional debiasing pipelines.
Significance. If the empirical claims were reliable, the paper would offer a useful practical contribution: a post-hoc, retraining-free debiasing toolkit with a unified evaluation metric. The study has positive features: it benchmarks multiple unlearning methods across three distinct bias types, includes privacy (MIA) and efficiency metrics, and provides qualitative GradCAM inspection. The paper also explicitly acknowledges in §4 that 'fairness gains may be superficial rather than substantive,' which shows a degree of self-awareness. However, the headline result is undermined by two load-bearing problems: (i) the 'minimal accuracy loss' claim is directly contradicted by the paper's own Table 2, and (ii) the fairness evaluation is confounded because D_f and the DP/EO partitions share the same subgroup axis, so the reported DP improvements may be a mechanical consequence of degrading forget-set predictions. In addition, the Co-BUM metric is not reproducible as defined, because the normalization N_X is never specified and the displayed formula is malformed. These issues prevent the current manuscript from supporting its central claims.
major comments (4)
- [Abstract, Table 2] The claim of 'minimal accuracy loss' is contradicted by the reported results. In Table 2, CelebA Gradient Ascent drops test accuracy from 94.20 to 50.90 (an absolute drop of 43.3 points), CUB LoRA drops from 78.69 to 57.47, and CUB Gradient Ascent drops to 64.41. Even methods with the best fairness gains show substantial utility loss. Since 'minimal accuracy loss' is a central selling point of the paper, this contradiction is load-bearing and the claim must be retracted or substantially qualified.
- [Figure 2, Table 1] Co-BUM, the proposed unified metric, is not reproducible as defined. Figure 2 defines Co-BUM in terms of a normalization N_X, but N_X is never defined in the paper or in Table 1, which only lists the weights α_i and scaling factor κ. Moreover, the displayed formula is garbled: 'Co-BUM := κ (Σ α_i) (Σ α_i / i)' is not a recognizable weighted harmonic mean, and the division and summation indices are ambiguous. Without a precise definition of N_X and the overall formula, the Co-BUM scores in Table 2 cannot be reproduced or compared across methods.
- [§3.1–3.3, §4] The fairness evaluation is confounded with the unlearning intervention. In each setting, D_f is defined as the biased subgroup (distant-pose birds, bird-with-patch, female-smiling faces), and DP/EO are computed on test partitions of the same subgroup axis. Unlearning methods such as Gradient Ascent are explicitly designed to maximize loss on D_f, so they directly change predictions on that group; the DP improvement may be a mechanical consequence of degrading forget-set performance rather than a genuine reduction in bias. The paper's own diagnostic in §4—'large [RA–TA] gaps indicate indiscriminate degradation'—applies to the CelebA Gradient Ascent result: retain accuracy drops from 96.13 to 58.31, a massive collapse, yet this method produces the headline 97.37% DP improvement. The reported gains therefore do not establish that unlearning debiases the model.
- [§3–§4 (overall design)] No experiment separates 'fairness improvement' from 'forgetting the subgroup.' To support the central claim, the authors would need an independent evaluation target: for example, measuring fairness on an attribute not used to construct D_f, or comparing against a control that randomly degrades predictions on D_f to show that the DP gain exceeds the mechanical effect. Without such a control, the causal claim that unlearning removes spurious correlations rather than simply damaging the forget set is unsupported.
minor comments (5)
- [Table 2] Several rows in Table 2 are typeset with concatenated values (e.g., the CUB Gradient Ascent row appears as 'Gradient Ascent3774.67 64.41 93.75 ...'), making the table difficult to read and potentially ambiguous. Please reformat.
- [§3.3] The statement that FMD is 'rendered irrelevant' due to lack of counterfactuals is terse; a more explicit rationale would help readers understand why the comparison is omitted in this setting.
- [References] Some references are incomplete or lack venue details (e.g., [27] has no conference/proceedings information). Please ensure all citations are fully specified.
- [Abstract] The phrase 'minimal accuracy loss' should be replaced with language that accurately reflects the utility-fairness trade-offs shown in Table 2, or the accuracy figures should be reported with confidence intervals to support the claim.
- [Figure 1 caption] The caption attributes the CIFAR-10 debiasing to 'the SCRUB framework,' while Section 2 describes a teacher-student method 'inspired by SCRUB.' Please clarify whether SCRUB is used exactly as in the original paper or as an adapted variant.
Circularity Check
Fairness evaluation is entangled with the unlearning target: D_f is defined as the biased subgroup, and DP/EO are measured on the same subgroup axis, so the reported fairness gains are partly mechanical consequences of forgetting D_f rather than evidence of genuine debiasing.
-
self definitional
[Section 3.1 and Section 3.4]
"The bird samples with pose bins 0 and 1 constitute Dr, and those with pose bin 2, the Df."
The unlearning target D_f is defined as one side of the exact partition that later serves as the fairness evaluation axis. Section 3.4 states that fairness is measured 'across demographic partitions (e.g., male vs. female or far vs. near)', and on CUB the 'far vs. near' partition is precisely pose bin 2 vs. bins 0/1. Gradient ascent, LoRA, and SCRUB are all explicitly optimized to change predictions on D_f, so the same samples used to define the bias intervention are also the samples on which DP/EO improvements are computed. The reported disparity reduction is therefore at least partly forced by the construction: the intervention set and the evaluation set coincide by definition.
-
fitted input called prediction
[Section 4, Table 2 (CelebA) and Section 4 General Trends]
"The RA-TA gap further serves as a diagnostic, small gaps signal targeted forgetting with minimal collateral damage, while large gaps indicate indiscriminate degradation."
Using the paper's own diagnostic, the CelebA gradient-ascent result (FA=3.39, RA=58.31, TA=50.90) is a clear case of a large RA-TA gap, i.e., 'indiscriminate degradation'. Yet this same result is the source of the headline 97.37% demographic-parity improvement. The DP gain is the mechanical outcome of maximizing loss on the forget set D_f, which overlaps with the female-smiling subgroup; the model is made worse on that subgroup, and the positive-rate gap narrows as a direct result. Calling this a fairness improvement is therefore predicting a quantity that is forced by the optimization target rather than measured on an independently defined fairness axis.
full rationale
The paper is not globally circular: the unlearning methods are standard and drawn from external work, the datasets are established benchmarks, and no load-bearing self-citation chain or imported uniqueness theorem is present. However, the central empirical claim—that post-hoc unlearning substantially reduces subgroup disparities—rests on an evaluation setup in which D_f and the fairness partitions share the same subgroup axis. On CUB, D_f is the distant-pose bin and fairness is reported on far-vs-near partitions; on CIFAR, D_f is the bird+patch set and fairness is reported on the same class/patch axis; on CelebA, D_f is the biased gender-smiling group and fairness is reported on gender partitions. Because the unlearning objective is explicitly to degrade performance on D_f, the DP/EO improvements are partly mechanical consequences of the construction. The paper's own RA-TA diagnostic further confirms that the largest headline DP gain (CelebA, 97.37%) comes from a regime it would classify as indiscriminate degradation. Thus the fairness improvement is not an independent confirmation of debiasing; it is, to a substantial degree, the expected output of the intervention by definition.
Assumptions & free parameters
free parameters (6)
- Co-BUM metric weights =
alpha_U=0.25, alpha_F=0.25, alpha_Q=1, alpha_P=1, alpha_E=1, kappa=1, gamma=0.5
- Gradient ascent retain-regularization weight alpha =
1
- LoRA forget weight beta =
1
- LoRA rank =
8 for CUB/CIFAR, 4 for CelebA
- Pose bin quantile thresholds for CUB =
Three quantile bins (0,1,2) over bounding-box area
- Red patch coverage fraction for CIFAR-10 =
50% of class-2 samples
assumptions (4)
- domain assumption Images can be decomposed as x = s + b, where s is semantic and b is spurious (Section 1)
- domain assumption ResNet-18 pretrained on ImageNet is a representative backbone for bias evaluation
- domain assumption Influence functions approximate the bias contribution for FMD (Section 2)
- domain assumption Demographic parity and equalized odds computed over the hand-defined partitions capture the relevant fairness
invented entities (1)
-
Co-BUM (Concerted Bias and Unlearning Metric)
Cite this review
Pith. "Pith review of Bias-Aware Machine Unlearning: Towards Fairer Vision Models via Controllable Forgetting." pith.science (2026). https://pith.science/paper/JA5UPLG7
@misc{pith2026250907456,
author = {Pith},
title = {Pith review of: Bias-Aware Machine Unlearning: Towards Fairer Vision Models via Controllable Forgetting},
year = {2026},
howpublished = {\url{https://pith.science/paper/JA5UPLG7}},
note = {Machine review of arXiv:2509.07456}
}
read the original abstract
Deep neural networks often rely on spurious correlations in training data, leading to biased or unfair predictions in safety-critical domains such as medicine and autonomous driving. While conventional bias mitigation typically requires retraining from scratch or redesigning data pipelines, recent advances in machine unlearning provide a promising alternative for post-hoc model correction. In this work, we investigate \textit{Bias-Aware Machine Unlearning}, a paradigm that selectively removes biased samples or feature representations to mitigate diverse forms of bias in vision models. Building on privacy-preserving unlearning techniques, we evaluate various strategies including Gradient Ascent, LoRA, and Teacher-Student distillation. Through empirical analysis on three benchmark datasets, CUB-200-2011 (pose bias), CIFAR-10 (synthetic patch bias), and CelebA (gender bias in smile detection), we demonstrate that post-hoc unlearning can substantially reduce subgroup disparities, with improvements in demographic parity of up to \textbf{94.86\%} on CUB-200, \textbf{30.28\%} on CIFAR-10, and \textbf{97.37\%} on CelebA. These gains are achieved with minimal accuracy loss and with methods scoring an average of 0.62 across the 3 settings on the joint evaluation of utility, fairness, quality, and privacy. Our findings establish machine unlearning as a practical framework for enhancing fairness in deployed vision systems without necessitating full retraining.
Figures
Reference graph
Works this paper leans on
-
[7]
Fast model debias with machine un- learning
Ruizhe Chen, Jianfei Yang, Huimin Xiong, Jianhong Bai, Tianxiang Hu, Jin Hao, Yang Feng, Joey Tianyi Zhou, Jian Wu, and Zuozhu Liu. Fast model debias with machine un- learning. InNeurIPS, 2023. 2, 3, 4
work page 2023
-
[20]
Fair machine unlearning: Data removal while mitigating disparities
Alex Oesterling, Jiaqi Ma, Flavio Calmon, and Himabindu Lakkaraju. Fair machine unlearning: Data removal while mitigating disparities. InInternational Conference on Artifi- cial Intelligence and Statistics. PMLR, 2024. 3
work page 2024
-
[1]
Recognition in terra incognita
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In(ECCV), 2018. 2
work page 2018
-
[2]
Ludovic Bourtoule, Varun Chandrasekaran, et al. Machine unlearning. InIEEE Symposium on Security and Privacy,
-
[3]
Bird species categorization using pose normalized deep convolutional nets, 2014
Steve Branson, Grant Van Horn, Serge Belongie, and Pietro Perona. Bird species categorization using pose normalized deep convolutional nets, 2014. 2
work page 2014
-
[4]
Gender shades: Inter- sectional accuracy disparities in commercial gender classifi- cation
Joy Buolamwini and Timnit Gebru. Gender shades: Inter- sectional accuracy disparities in commercial gender classifi- cation. InConference on fairness, accountability and trans- parency. PMLR, 2018. 2
work page 2018
-
[5]
Building classifiers with independency constraints
Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. Building classifiers with independency constraints. InICDM Workshops, 2009. 5
work page 2009
-
[6]
When machine unlearn- ing jeopardizes privacy
Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. When machine unlearn- ing jeopardizes privacy. InACM SIGSAC, 2021. 5
work page 2021
Show all 29 references
-
[8]
Fairness and bias mitigation in com- puter vision: A survey, 2024
Sepehr Dehdashtian, Ruozhen He, Yi Li, Guha Bal- akrishnan, Nuno Vasconcelos, Vicente Ordonez, and Vishnu Naresh Boddeti. Fairness and bias mitigation in com- puter vision: A survey, 2024. 2
2024
-
[9]
Can machine unlearning reduce social bias in lan- guage models? InEMNLP, pages 954–969, 2024
Omkar Dige, Diljot Arneja, Tsz Fung Yau, Qixuan Zhang, Mohammad Bolandraftar, Xiaodan Zhu, and Faiza Khan Khattak. Can machine unlearning reduce social bias in lan- guage models? InEMNLP, pages 954–969, 2024. 2
2024
-
[10]
Shortcut learning in deep neural networks
Robert Geirhos, J ¨orn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Fe- lix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020. 1, 2
2020
-
[11]
Making ai forget you: Data deletion in ma- chine learning.NeurIPS, 2019
Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. Making ai forget you: Data deletion in ma- chine learning.NeurIPS, 2019. 2
2019
-
[12]
Equality of opportunity in supervised learning
Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In(NeurIPS), 2016. 5
2016
-
[13]
Deep residual learning for image recognition, 2015
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015. 4
2015
-
[14]
SAP: corrective machine unlearning with scaled activation projection for label noise robustness
Sangamesh Kodge, Deepak Ravikumar, Gobinda Saha, and Kaushik Roy. SAP: corrective machine unlearning with scaled activation projection for label noise robustness. In AAAI, 2025. 2
2025
-
[15]
”alexa, can you forget me?” machine unlearn- ing benchmark in spoken language understanding, 2025
Alkis Koudounas, Claudio Savelli, Flavio Giobergia, and Elena Baralis. ”alexa, can you forget me?” machine unlearn- ing benchmark in spoken language understanding, 2025. 3, 5
2025
-
[16]
Towards unbounded machine unlearning
Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning. InNeurIPS, 2023. 3
2023
-
[17]
Deep learning face attributes in the wild
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In(ICCV), 2015. 2, 4
2015
-
[18]
Breaking the trilemma of privacy, utility, and efficiency via controllable machine unlearning
Zheyuan Liu, Guangyao Dou, Eli Chien, Chunhui Zhang, Yijun Tian, and Ziwei Zhu. Breaking the trilemma of privacy, utility, and efficiency via controllable machine unlearning. In WWW. ACM, 2024. 5
2024
-
[19]
A survey on bias and fairness in machine learning.ACM computing surveys (CSUR), 54 (6):1–35, 2021
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning.ACM computing surveys (CSUR), 54 (6):1–35, 2021. 1, 2
2021
-
[21]
Smith, and Chiyuan Zhang
Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, and Chiyuan Zhang. MUSE: machine un- learning six-way evaluation for language models. InICLR,
-
[22]
Don’t judge an object by its context: learning to overcome contex- tual bias
Krishna Kumar Singh, Dhruv Mahajan, Kristen Grauman, Yong Jae Lee, Matt Feiszli, and Deepti Ghadiyaram. Don’t judge an object by its context: learning to overcome contex- tual bias. InCVPR, 2020. 2
2020
-
[23]
Unrolling SGD: understanding factors influencing machine unlearning.CoRR, abs/2109.13398,
Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling SGD: understanding factors influencing machine unlearning.CoRR, abs/2109.13398,
-
[24]
Unbiased look at dataset bias
Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. InCVPR, 2011. 1
2011
-
[25]
To- wards fairness in visual recognition: Effective strategies for bias mitigation
Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. To- wards fairness in visual recognition: Effective strategies for bias mitigation. InCVPR, 2020. 2
2020
-
[26]
Don’t forget too much: Towards machine unlearning on feature level.IEEE Trans
Heng Xu, Tianqing Zhu, Wanlei Zhou, and Wei Zhao. Don’t forget too much: Towards machine unlearning on feature level.IEEE Trans. Dependable Secur. Comput., 22(2):1313– 1328, 2025. 2
2025
-
[27]
Facts: First amplify correlations and then slice to discover bias, 2023
Sriram Yenamandra, Pratik Ramesh, Viraj Prabhu, and Judy Hoffman. Facts: First amplify correlations and then slice to discover bias, 2023. 2
2023
-
[28]
Mitigating unwanted biases with adversarial learning
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In AIES. ACM, 2018. 2
2018
-
[29]
Geniu: A restricted data access unlearn- ing for imbalanced data, 2024
Chenhao Zhang, Shaofei Shen, Yawen Zhao, Weitong Tony Chen, and Miao Xu. Geniu: A restricted data access unlearn- ing for imbalanced data, 2024. 2
2024
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.