REVIEW 4 major objections 5 minor 22 references
FairDD: Enhancing Fairness with domain-incremental learning in dermatological disease diagnosis
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that FairDD, a domain-incremental learning method with replay, mixup, supervised contrastive learning, distillation fine-tuning, and a statistical-parity loss, achieves the best accuracy–fairness trade-off on two…
desk verdict A reasonable engineering combination that needs the loss definition and tuning protocol fixed before the empirical claim can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a domain-incremental training loop with a bounded reservoir replay buffer, combined with four auxiliary objectives. The overall loss is $L = L_{\mathrm{ce}} + L_{\mathrm{sup}} + \alpha L_{\mathrm{dis}} + \beta L_{\mathrm{spd}}$, where $L_{\mathrm{sup}}$ is supervised contrastive loss on projected embeddings, $L_{\mathrm{dis}}$ is teacher–student distillation fine-tuning on buffer data, and $L_{\mathrm{spd}}$ penalizes statistical parity disparity across sensitive groups. Cross-domain mixup, Eq. (1), creates convex combinations of new-domain and old-domain samples so the model cannot simply memorize the current domain. The trade-off is measured by the FAT(E) criterion, which combines relative accuracy change and relative fairness change against a chosen baseline model.
What would settle it
Implement Eq. (3) literally during training: each term contains an expectation over $\mathbb{1}\{q(X_i)=y\}$, a step function whose gradient is zero almost everywhere, so backpropagation cannot change the network through this loss. If the reported fairness gains persist with this literal implementation, the gains come from the other components; if they disappear, the statistical-parity term as written is not the mechanism, and the paper's stated objective must be a different, unstated function.
Extended reading notes
Core claim
FairDD's central claim is that organizing training as incremental exposure to demographic domains, rather than reweighting data or post-processing predictions, yields a better accuracy–fairness operating point. At each stage the student network trains on current-domain samples together with replayed old-domain samples; cross-domain mixup interpolates between them, supervised contrastive loss pulls same-class embeddings together, and a fixed teacher distills old knowledge while the model fine-tunes new knowledge. A statistical parity disparity term, Eq. (3), is added to make prediction rates of the target class equal across sensitive groups. On Fitzpatrick-17k, FairDD reports the lowest equal-opportunity and equal-odds values among all baselines and the highest FAT(E) trade-off scores; on ISIC 2019 it reports the best trade-off on all but one criterion, with the remaining criterion 0.02 above the next best. The paper concludes that FairDD significantly improves fairness while maintaining diagnostic performance, and it acknowledges an accuracy gap to the vanilla model.
Load-bearing premise
The fairness gains rest on the statistical-parity disparity term in Eq. (3) being an optimizable training objective, but as printed it averages hard indicators of predicted labels, which give the network no gradient to update from.
Editorial extensions
If this is right
- On Fitzpatrick-17k, FairDD lowers EOpp0 from 1.00 to 0.48 and EOdd from 10.54 to 5.65 while keeping accuracy at 86.53 versus 87.53 for vanilla, a drop of about one point.
- On ISIC 2019, FairDD reports EOpp0 0.59, EOpp1 4.58, and EOdd 4.21, improving fairness over vanilla while staying within roughly two accuracy points of it.
- Training order matters: the dark-to-light domain order improves both accuracy and fairness, while light-to-dark reverses the benefit, so the paper's results are tied to that schedule.
- Ablations show the best trade-off occurs at distillation weight $\alpha=0.6$ and that fairness first improves then declines as the statistical-parity weight $\beta$ increases.
- Larger replay buffers improve EOpp1 and EOdd with diminishing returns after capacity around 300.
Reading between the lines
- Not tested in the paper: replacing the hard-indicator statistical-parity term with a differentiable surrogate would show whether the reported fairness gains come from that loss or from the replay and contrastive components.
- Because the method only uses group membership to define domains, the same recipe should transfer to other medical imaging tasks and protected attributes such as sex or ethnicity in radiology.
- The sensitivity to domain order implies the method's fairness behavior is schedule-dependent, so real deployments would need to fix and document the domain ordering.
- The FAT(E) trade-off scores are relative to a specific baseline model, so they are not comparable across papers unless the same vanilla backbone and denominator are used.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FairDD, a dermatological disease diagnosis model that combines domain-incremental learning with cross-domain mixup, supervised contrastive learning, distillation fine-tuning, and a statistical parity disparity loss. Sensitive attribute groups are treated as domains presented sequentially, a fixed-size replay buffer is used to mitigate catastrophic forgetting, and the model is optimized with the weighted sum in Eq. (6). Experiments on Fitzpatrick-17k and ISIC 2019 compare FairDD with existing fairness methods on accuracy/performance metrics, EOpp/EOdd fairness metrics, and the FATE trade-off criterion; ablations cover training order, mixup, contrastive loss, distillation weight α, parity weight β, and buffer size. The paper claims that FairDD achieves the best accuracy-fairness trade-off among the compared methods.
Significance. If the reported results are reproducible, the paper makes a useful contribution: it shows that an incremental-learning framework with replay, mixup, contrastive learning, and distillation can improve fairness metrics on two medical imaging datasets while keeping accuracy close to the vanilla model, and it directly compares against relevant baselines such as FairAdaBN and QP-Net. A strength of the paper is that each proposed component is examined in an ablation study, including training order, mixup, contrastive loss, α, β, and buffer size. However, the current manuscript does not make the central empirical claim verifiable: the stated parity loss in Eq. (3) is not differentiable as written, the main experimental configuration is ambiguous due to conflicting statements about α, and the trade-off results in Table I are reported without standard deviations or significance tests. These are fixable in revision, so the direction is promising, but the evidence as presented is not yet sufficient for the paper's claims.
major comments (4)
- [II-D, Eq. (3); II-F, Eq. (6)] The statistical parity disparity loss in Eq. (3) is written with hard indicators 1{q(X_i)=y}. These indicators are piecewise constant and have zero gradient almost everywhere, so the loss cannot be minimized by gradient descent as printed. No surrogate loss, straight-through estimator, or soft probability relaxation is described in Section II or III. Because L_spd enters the total objective in Eq. (6) with weight β, the fairness improvements in Table I and the β ablation in Section III-C cannot be attributed to the stated equation. Please define the exact differentiable objective used in the implementation, or replace Eq. (3) with an explicit differentiable surrogate and justify its use.
- [Table I] The FATE trade-off columns E0, E1, and E2 are reported as point estimates without standard deviations, although the accuracy and fairness columns in the same table include mean±std. The paper's central claim that FairDD 'achieves the highest values in each trade-off criterion' rests on these point estimates. Please report mean±std for E0-E2 over the same multiple seeds and provide a significance test or confidence intervals to support the claimed ranking.
- [III and III-C] There is a direct inconsistency in the value of α: Section III states that α=1 for the main experiments, while Section III-C reports that the best fairness and trade-off values are obtained at α=0.6 and then sets α=0.6 for the statistical parity loss ablation. This makes it unclear which configuration produced Table I. Please specify the exact hyperparameters used for every reported result and describe a validation protocol for α and β that does not select values directly on the test metrics used in the final comparison.
- [III and III-C] Several key choices appear to be tuned on the same target metrics used for evaluation: β is set per dataset (0.6 for Fitzpatrick-17k, 1 for ISIC 2019), the training order is selected after observing the fairness/trade-off results in Table II, and the α, β, and buffer-size ablations are evaluated on the same trade-off criterion reported as the main result. Without a separate validation set or nested selection procedure, the reported comparisons risk overfitting to the test set. Please describe the validation and model-selection protocol in detail.
minor comments (5)
- [II-D, Eq. (3)] The notation "EXi∼SA=0 |(q(Xi)=y)" in Eq. (3) is malformed; it should be written, for example, as E_{X~P_{A=0}}[1{q(X)=y}].
- [III-C] There are cross-reference errors in Section III-C: the distillation fine-tuning ablation refers to Fig. 3b/3c but should refer to Fig. 2b/2c; the statistical parity loss ablation refers to Fig. 4a but should refer to Fig. 3a; and the buffer size ablation refers to Fig. 5a, while the paper only contains Figs. 2-4.
- [Table I] The formatting of values such as "86.530.14" should use explicit plus-minus signs (e.g., 86.53 ± 0.14) to avoid ambiguity.
- [References] The reference cited for the ISIC 2019 dataset, [21], is the ISBI 2017 challenge paper; please cite the actual ISIC 2019 challenge dataset or clarify why this reference is used.
- [II-E, Eq. (8)] Equation (8) is labeled "FATEFC" but the subscript FC is not defined in the text; please define FC as the fairness criterion used in that formula.
Circularity Check
No significant circularity: FairDD's claims are empirical comparisons against external baselines, not derivations that reduce to their own inputs.
full rationale
FairDD is an empirical systems paper: its central claim (a better accuracy-fairness trade-off than FairAdaBN, QP-Net, GroupDRO, etc.) is supported by Table I, which compares against external baselines on Fitzpatrick-17k and ISIC 2019. The training objective in Eq. (6) is a sum of standard cross-entropy, supervised contrastive loss (Eq. 2), distillation loss (Eq. 4), and a parity regularizer (Eq. 3); none of these terms is defined in terms of the evaluation metrics (EOpp, EOdd, FAT) in a way that makes the reported numbers true by construction. In fact, the parity loss targets statistical parity whereas the evaluation uses equal opportunity and equal odds, so the claimed fairness gains are not self-fulfilling. No author self-citations are used as load-bearing support; references such as [7], [10], and [16] are external prior work. Two weaknesses—the non-differentiable hard indicator in Eq. (3) and the apparent selection of hyperparameters (α, β, training order) after inspecting the same trade-off metrics—are correctness and reproducibility concerns, not circularity: they do not amount to a fitted parameter being renamed as a prediction, nor to an equation reducing to its input. The central comparison is against external benchmarks and could in principle disconfirm the paper's claim, so the circularity score is 0.
Assumptions & free parameters
free parameters (7)
- Mixup interpolation parameter theta =
theta = 0.8
- Supervised contrastive temperature tau =
tau = 0.07
- Distillation temperature T =
T = 2
- Distillation loss weight alpha =
alpha = 1 in main experiments, alpha = 0.6 in ablation
- Statistical parity disparity loss weight beta =
beta = 0.6 for Fitzpatrick-17k, beta = 1 for ISIC 2019, beta = 1 in buffer ablation
- Replay buffer size B =
B = 300
- FATE tradeoff parameter lambda =
lambda = 1.0
assumptions (4)
- domain assumption Reservoir replay of B samples preserves old-domain knowledge enough to prevent catastrophic forgetting.
- domain assumption Same-class embeddings should align across domains, making distillation on buffer samples a valid way to retain old knowledge.
- ad hoc to paper The hard-indicator statistical parity loss in Eq. (3) can be optimized by gradient descent.
- ad hoc to paper Summing cross-entropy, SupCon, distillation, and parity losses with fixed weights does not cause harmful gradient interference.
Cite this review
Pith. "Pith review of FairDD: Enhancing Fairness with domain-incremental learning in dermatological disease diagnosis." pith.science (2026). https://pith.science/paper/TDOYTFLN
@misc{pith2026241216542,
author = {Pith},
title = {Pith review of: FairDD: Enhancing Fairness with domain-incremental learning in dermatological disease diagnosis},
year = {2026},
howpublished = {\url{https://pith.science/paper/TDOYTFLN}},
note = {Machine review of arXiv:2412.16542}
}
read the original abstract
With the rapid advancement of deep learning technologies, artificial intelligence has become increasingly prevalent in the research and application of dermatological disease diagnosis. However, this data-driven approach often faces issues related to decision bias. Existing fairness enhancement techniques typically come at a substantial cost to accuracy. This study aims to achieve a better trade-off between accuracy and fairness in dermatological diagnostic models. To this end, we propose a novel fair dermatological diagnosis network, named FairDD, which leverages domain incremental learning to balance the learning of different groups by being sensitive to changes in data distribution. Additionally, we incorporate the mixup data augmentation technique and supervised contrastive learning to enhance the network's robustness and generalization. Experimental validation on two dermatological datasets demonstrates that our proposed method excels in both fairness criteria and the trade-off between fairness and performance.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
S. K. Zhou, H. Greenspan, C. Davatzikos, J. S. Duncan, B. Van Gin- neken, A. Madabhushi, J. L. Prince, D. Rueckert, and R. M. Summers, “A review of deep learning in medical imaging: Imaging traits, technol- ogy trends, case studies with progress highlights, and future promises,” Proceedings of the IEEE , vol. 109, no. 5, pp. 820–838, 2021
work page 2021
-
[2]
S. Yao, F. Dai, P. Sun, W. Zhang, B. Qian, and H. Lu, “Enhancing the fairness of ai prediction models by quasi-pareto improvement among heterogeneous thyroid nodule population,” Nature Communications , vol. 15, no. 1, pp. 1–13, 2024
work page 2024
-
[3]
Medfair: Benchmarking fairness for medical imaging,
Y . Zong, Y . Yang, and T. M. Hospedales, “Medfair: Benchmarking fairness for medical imaging,” in The Eleventh International Conference on Learning Representations , 2023
work page 2023
-
[4]
P. Seth and A. K. Pai, “Does the fairness of your pre-training hold up? examining the influence of pre-training techniques on skin tone bias in skin lesion classification,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 570–577
work page 2024
-
[5]
Fairprune: Achieving fairness through pruning for dermatological disease diagnosis,
Y . Wu, D. Zeng, X. Xu, Y . Shi, and J. Hu, “Fairprune: Achieving fairness through pruning for dermatological disease diagnosis,” in International Conference on Medical Image Computing and Computer-Assisted Inter- vention. Springer, 2022, pp. 743–753
work page 2022
-
[6]
A survey on bias and fairness in machine learning,
N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM computing surveys (CSUR), vol. 54, no. 6, pp. 1–35, 2021
2021
-
[7]
Z. Xu, S. Zhao, Q. Quan, Q. Yao, and S. K. Zhou, “Fairadabn: Miti- gating unfairness with adaptive batch normalization and its application to dermatological disease classification,” in International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2023, pp. 307–317
work page 2023
-
[8]
E. Puyol-Ant ´on, B. Ruijsink, S. K. Piechnik, S. Neubauer, S. E. Petersen, R. Razavi, and A. P. King, “Fairness in cardiac mr image analysis: an investigation of bias due to data imbalance in deep learning based segmentation,” in Medical Image Computing and Computer Assisted Intervention. Springer, 2021, pp. 413–423
work page 2021
Show all 22 references
-
[9]
Improving fairness of automated chest x-ray diagnosis by contrastive learning,
M. Lin, T. Li, Z. Sun, G. Holste, Y . Ding, F. Wang, G. Shih, and Y . Peng, “Improving fairness of automated chest x-ray diagnosis by contrastive learning,” arXiv preprint arXiv:2401.15111 , 2024
2024 arXiv
-
[10]
Domain-incremental continual learning for mitigating bias in facial expression and action unit recog- nition,
N. Churamani, O. Kara, and H. Gunes, “Domain-incremental continual learning for mitigating bias in facial expression and action unit recog- nition,” IEEE Transactions on Affective Computing , 2022
2022
-
[11]
Catastrophic interference in connec- tionist networks: The sequential learning problem,
M. McCloskey and N. J. Cohen, “Catastrophic interference in connec- tionist networks: The sequential learning problem,” in Psychology of learning and motivation . Elsevier, 1989, vol. 24, pp. 109–165
1989
-
[12]
Class-incremental continual learning into the extended der-verse,
M. Boschini, L. Bonicelli, P. Buzzega, A. Porrello, and S. Calderara, “Class-incremental continual learning into the extended der-verse,”IEEE transactions on pattern analysis and machine intelligence , vol. 45, no. 5, pp. 5497–5512, 2022
2022
-
[13]
mixup: Beyond empirical risk minimization,
H. Zhang, M. Cisse, Y . N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” arXiv preprint arXiv:1710.09412 , 2017
2017 arXiv
-
[14]
Supervised contrastive learn- ing,
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y . Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, “Supervised contrastive learn- ing,” Advances in neural information processing systems , vol. 33, pp. 18 661–18 673, 2020
2020
-
[15]
Learning fair representations,
R. Zemel, Y . Wu, K. Swersky, T. Pitassi, and C. Dwork, “Learning fair representations,” in International conference on machine learning . PMLR, 2013, pp. 325–333
2013
-
[16]
Distributionally robust neural networks for group shifts: On the importance of regular- ization for worst-case generalization,
S. Sagawa, P. W. Koh, T. B. Hashimoto, and P. Liang, “Distributionally robust neural networks for group shifts: On the importance of regular- ization for worst-case generalization,” arXiv preprint arXiv:1911.08731 , 2019
1911 arXiv
-
[17]
End: Entangling and disentangling deep representations for bias correction,
E. Tartaglione, C. A. Barbano, and M. Grangetto, “End: Entangling and disentangling deep representations for bias correction,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 13 508–13 517
2021
-
[18]
Conditional learning of fair representations,
H. Zhao, A. Coston, T. Adel, and G. J. Gordon, “Conditional learning of fair representations,” arXiv preprint arXiv:1910.07162 , 2019
1910 arXiv
-
[19]
Pcr: Proxy-based contrastive replay for online class-incremental continual learning,
H. Lin, B. Zhang, S. Feng, X. Li, and Y . Ye, “Pcr: Proxy-based contrastive replay for online class-incremental continual learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 24 246–24 255
2023
-
[20]
Towards transparency in dermatology image datasets with skin tone annotations by experts, crowds, and an algorithm,
M. Groh, C. Harris, R. Daneshjou, O. Badri, and A. Koochek, “Towards transparency in dermatology image datasets with skin tone annotations by experts, crowds, and an algorithm,” Proceedings of the ACM on Human-Computer Interaction , vol. 6, no. CSCW2, pp. 1–26, 2022
2022
-
[21]
N. C. Codella, D. Gutman, M. E. Celebi, B. Helba, M. A. Marchetti, S. W. Dusza, A. Kalloo, K. Liopyris, N. Mishra, H. Kittler et al. , “Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the i...
2017
-
[22]
Equality of opportunity in supervised learning,
M. Hardt, E. Price, and N. Srebro, “Equality of opportunity in supervised learning,” Advances in neural information processing systems , vol. 29, 2016
2016
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.