{"id":"100743e7-5a06-4f25-9d41-b663f043e298","arxiv_id":"2412.16542","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"FairDD uses domain-incremental replay, mixup, contrastive learning, and distillation to improve the accuracy-fairness tradeoff of dermatological classifiers on Fitzpatrick-17k and ISIC 2019.","lead":"The authors built a skin-disease AI that learns each patient group one stage at a time, mixing new images with remembered old ones and adding penalties so predictions are more equal across skin tone or age. On two public datasets it reports better fairness at a modest accuracy cost, but the evidence has important gaps.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3)'s parity loss is non-differentiable as written; without a stated surrogate, the fairness gains in Table I and the ablation cannot be attributed to the proposed objective.","rationale":"I read FairDD as a combination of known components whose central empirical claim is that the stated objective improves fairness while maintaining diagnostic performance. For that claim to hold, every term in Eq. (6) must be optimizable and the reported configuration must be identifiable. The weakest link is Eq. (3): it uses hard indicators of predicted labels, which yield zero gradient almost everywhere. The paper's own ablation credits this loss for the fairness improvement, so the claim hinges on an implementation detail the text does not provide. The reader's conditional verdict is appropriate; I would not reject outright because a soft relaxation could repair the paper. The alpha=1 main setting versus the alpha=0.6 ablation optimum, the selection of training order after seeing results, and the per-dataset beta choices are additional reproducibility concerns, but the non-differentiability of the parity loss is the clearest single load-bearing point. The paper offers no independent supporting evidence such as code or formal verification, so the reported numbers cannot currently be checked against the stated method.","tokens_in":8505,"tokens_out":6843,"duration_ms":63307,"concrete_test":"Instrument Eq. (3) in a reimplementation of FairDD: compute the gradient norm ||dL_spd/dtheta|| over one training batch with the hard-indicator loss as printed. If it is zero almost everywhere, the stated objective cannot drive fairness improvements; then replace the indicator with a soft probability surrogate (q_y(X) in place of 1{q(X)=y}), rerun the Fitzpatrick-17k ablation, and compare the fairness metrics and trade-off scores with the published values in Table I and the parity-loss ablation. If the soft-surrogate results differ beyond experimental noise, the reported attribution to Eq. (3) is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that FairDD's statistical parity disparity loss is a key mechanism for improving the accuracy-fairness trade-off—requires Eq. (3) to be an optimizable training objective. As printed, L_spd = sum_y || E_{X~A=0}[1{q(X)=y}] - E_{X~A=1}[1{q(X)=y}] ||_2 uses hard indicators of the predicted class. These indicators are piecewise constant and have zero gradient almost everywhere, and the paper provides no surrogate loss, straight-through estimator, or soft relaxation. The ablation attributed to this loss (Section III-C, 'Impact of statistical parity disparity loss') claims it improves fairness, and the main results in Table I include it through beta in Eq. (6). If the implementation actually replaces the hard indicator with the soft probability q_y(X), the text does not define this, so the reported numbers cannot be reproduced or traced to the stated mechanism. A compounding reproducibility issue is that Section III sets alpha=1 for the main experiments, while the ablation reports the best trade-off at alpha=0.6; without a validation protocol, it is unclear which configuration produced Table I. These issues are fixable, but as written the empirical claim is not verifiable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FairDD, a dermatological disease diagnosis model that combines domain-incremental learning with cross-domain mixup, supervised contrastive learning, distillation fine-tuning, and a statistical parity disparity loss. Sensitive attribute groups are treated as domains presented sequentially, a fixed-size replay buffer is used to mitigate catastrophic forgetting, and the model is optimized with the weighted sum in Eq. (6). Experiments on Fitzpatrick-17k and ISIC 2019 compare FairDD with existing fairness methods on accuracy/performance metrics, EOpp/EOdd fairness metrics, and the FATE trade-off criterion; ablations cover training order, mixup, contrastive loss, distillation weight α, parity weight β, and buffer size. The paper claims that FairDD achieves the best accuracy-fairness trade-off among the compared methods.","tokens_in":8805,"tokens_out":5671,"duration_ms":62105,"significance":"If the reported results are reproducible, the paper makes a useful contribution: it shows that an incremental-learning framework with replay, mixup, contrastive learning, and distillation can improve fairness metrics on two medical imaging datasets while keeping accuracy close to the vanilla model, and it directly compares against relevant baselines such as FairAdaBN and QP-Net. A strength of the paper is that each proposed component is examined in an ablation study, including training order, mixup, contrastive loss, α, β, and buffer size. However, the current manuscript does not make the central empirical claim verifiable: the stated parity loss in Eq. (3) is not differentiable as written, the main experimental configuration is ambiguous due to conflicting statements about α, and the trade-off results in Table I are reported without standard deviations or significance tests. These are fixable in revision, so the direction is promising, but the evidence as presented is not yet sufficient for the paper's claims.","major_comments":[{"comment":"The statistical parity disparity loss in Eq. (3) is written with hard indicators 1{q(X_i)=y}. These indicators are piecewise constant and have zero gradient almost everywhere, so the loss cannot be minimized by gradient descent as printed. No surrogate loss, straight-through estimator, or soft probability relaxation is described in Section II or III. Because L_spd enters the total objective in Eq. (6) with weight β, the fairness improvements in Table I and the β ablation in Section III-C cannot be attributed to the stated equation. Please define the exact differentiable objective used in the implementation, or replace Eq. (3) with an explicit differentiable surrogate and justify its use.","section":"II-D, Eq. (3); II-F, Eq. (6)"},{"comment":"The FATE trade-off columns E0, E1, and E2 are reported as point estimates without standard deviations, although the accuracy and fairness columns in the same table include mean±std. The paper's central claim that FairDD 'achieves the highest values in each trade-off criterion' rests on these point estimates. Please report mean±std for E0-E2 over the same multiple seeds and provide a significance test or confidence intervals to support the claimed ranking.","section":"Table I"},{"comment":"There is a direct inconsistency in the value of α: Section III states that α=1 for the main experiments, while Section III-C reports that the best fairness and trade-off values are obtained at α=0.6 and then sets α=0.6 for the statistical parity loss ablation. This makes it unclear which configuration produced Table I. Please specify the exact hyperparameters used for every reported result and describe a validation protocol for α and β that does not select values directly on the test metrics used in the final comparison.","section":"III and III-C"},{"comment":"Several key choices appear to be tuned on the same target metrics used for evaluation: β is set per dataset (0.6 for Fitzpatrick-17k, 1 for ISIC 2019), the training order is selected after observing the fairness/trade-off results in Table II, and the α, β, and buffer-size ablations are evaluated on the same trade-off criterion reported as the main result. Without a separate validation set or nested selection procedure, the reported comparisons risk overfitting to the test set. Please describe the validation and model-selection protocol in detail.","section":"III and III-C"}],"minor_comments":[{"comment":"The notation \"EXi∼SA=0 |(q(Xi)=y)\" in Eq. (3) is malformed; it should be written, for example, as E_{X~P_{A=0}}[1{q(X)=y}].","section":"II-D, Eq. (3)"},{"comment":"There are cross-reference errors in Section III-C: the distillation fine-tuning ablation refers to Fig. 3b/3c but should refer to Fig. 2b/2c; the statistical parity loss ablation refers to Fig. 4a but should refer to Fig. 3a; and the buffer size ablation refers to Fig. 5a, while the paper only contains Figs. 2-4.","section":"III-C"},{"comment":"The formatting of values such as \"86.530.14\" should use explicit plus-minus signs (e.g., 86.53 ± 0.14) to avoid ambiguity.","section":"Table I"},{"comment":"The reference cited for the ISIC 2019 dataset, [21], is the ISBI 2017 challenge paper; please cite the actual ISIC 2019 challenge dataset or clarify why this reference is used.","section":"References"},{"comment":"Equation (8) is labeled \"FATEFC\" but the subscript FC is not defined in the text; please define FC as the fairness criterion used in that formula.","section":"II-E, Eq. (8)"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and the proposed combination of ideas is interesting, but the current evidence has three load-bearing gaps: the stated parity loss is non-differentiable, the main hyperparameter configuration is ambiguous, and the trade-off claims lack variance estimates. These are fixable in a major revision; I do not see a reason for rejection at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: FairDD is a competent combination of known continual-learning and fairness ingredients applied to dermatology. It is not a conceptual breakthrough, but it is a sensible pipeline and the ablation study is more thorough than many papers at this level. The main results are plausible, but the paper as written has a load-bearing technical gap: the statistical parity disparity loss (Eq. 3) is not differentiable as stated.\n\nThe new part is the specific assembly: domain-incremental training with replay, cross-domain mixup, supervised contrastive loss, distillation fine-tuning, and a parity disparity term. Each component exists, but putting them together for fairness on Fitzpatrick-17k and ISIC 2019 is new, and the empirical comparison against other fairness methods is useful. The ablations on training order, mixup, contrastive loss, distillation weight, and buffer size give a reasonably complete picture of what matters.\n\nThe soft spots are real but fixable. Eq. (3) uses hard indicators of the predicted class, which have zero gradient almost everywhere. The authors likely mean a soft relaxation using predicted probabilities, but the text doesn't say so. Without that, the reported fairness gains cannot be traced to the stated objective. Compounding that, Section III says alpha=1 for the main experiments, while the distillation ablation finds the best trade-off at alpha=0.6. No validation protocol is given, so it is unclear which configuration produced Table I. The FATE trade-off columns also lack standard deviations, and beta is tuned per dataset on the target metrics. None of this proves the method is wrong, but it makes the central empirical claim hard to verify as written.\n\nFor whom: researchers working on fairness in medical imaging will find this a useful starting point, especially the training-order results. But the paper needs a revision that clarifies the loss, reports the actual hyperparameter settings, adds error bars on FATE, and ideally releases code, before the results can be used with confidence. I would send it to peer review rather than desk reject, because the empirical direction is plausible and the flaws are correctable.","headline":"A reasonable engineering combination that needs the loss definition and tuning protocol fixed before the empirical claim can be trusted.","tokens_in":9270,"tokens_out":2744,"would_cite":false,"duration_ms":22298,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that FairDD, a domain-incremental learning method with replay, mixup, supervised contrastive learning, distillation fine-tuning, and a statistical-parity loss, achieves the best accuracy–fairness trade-off on two…","keywords":["fairness","domain-incremental learning","dermatological disease diagnosis","statistical parity disparity","mixup data augmentation","supervised contrastive learning","knowledge distillation","accuracy-fairness trade-off"],"falsifier":"Implement Eq. (3) literally during training: each term contains an expectation over $\\mathbb{1}\\{q(X_i)=y\\}$, a step function whose gradient is zero almost everywhere, so backpropagation cannot change the network through this loss. If the reported fairness gains persist with this literal implementation, the gains come from the other components; if they disappear, the statistical-parity term as written is not the mechanism, and the paper's stated objective must be a different, unstated function.","tokens_in":8291,"feed_emoji":"⚖️","tokens_out":8160,"duration_ms":63987,"temperature":0.7,"pith_summary":"This paper tries to establish that fairness in dermatological diagnosis does not have to cost accuracy. It introduces FairDD, a network trained with domain-incremental learning where each sensitive-attribute group (skin type or age) is treated as a domain, and adds memory replay, cross-domain mixup, supervised contrastive learning, distillation fine-tuning, and a statistical-parity disparity loss. The authors report that FairDD achieves the best fairness values and the best accuracy–fairness trade-off scores among all compared methods on Fitzpatrick-17k and ISIC 2019, reducing equal-opportunity and equal-odds disparities substantially while keeping accuracy within a few points of the vanilla model. If true, this gives a recipe for deploying fairer skin-lesion classifiers without the large accuracy penalty typical of earlier fairness methods.","feed_headline":"FairDD: fairer skin-lesion AI with little accuracy loss","feed_subtitle":"Memory replay, mixup, contrastive learning, and distillation beat fairness baselines on Fitzpatrick-17k and ISIC 2019.","key_machinery":"The load-bearing mechanism is a domain-incremental training loop with a bounded reservoir replay buffer, combined with four auxiliary objectives. The overall loss is $L = L_{\\mathrm{ce}} + L_{\\mathrm{sup}} + \\alpha L_{\\mathrm{dis}} + \\beta L_{\\mathrm{spd}}$, where $L_{\\mathrm{sup}}$ is supervised contrastive loss on projected embeddings, $L_{\\mathrm{dis}}$ is teacher–student distillation fine-tuning on buffer data, and $L_{\\mathrm{spd}}$ penalizes statistical parity disparity across sensitive groups. Cross-domain mixup, Eq. (1), creates convex combinations of new-domain and old-domain samples so the model cannot simply memorize the current domain. The trade-off is measured by the FAT(E) criterion, which combines relative accuracy change and relative fairness change against a chosen baseline model.","core_discovery":"FairDD's central claim is that organizing training as incremental exposure to demographic domains, rather than reweighting data or post-processing predictions, yields a better accuracy–fairness operating point. At each stage the student network trains on current-domain samples together with replayed old-domain samples; cross-domain mixup interpolates between them, supervised contrastive loss pulls same-class embeddings together, and a fixed teacher distills old knowledge while the model fine-tunes new knowledge. A statistical parity disparity term, Eq. (3), is added to make prediction rates of the target class equal across sensitive groups. On Fitzpatrick-17k, FairDD reports the lowest equal-opportunity and equal-odds values among all baselines and the highest FAT(E) trade-off scores; on ISIC 2019 it reports the best trade-off on all but one criterion, with the remaining criterion 0.02 above the next best. The paper concludes that FairDD significantly improves fairness while maintaining diagnostic performance, and it acknowledges an accuracy gap to the vanilla model.","pith_inferences":["Not tested in the paper: replacing the hard-indicator statistical-parity term with a differentiable surrogate would show whether the reported fairness gains come from that loss or from the replay and contrastive components.","Because the method only uses group membership to define domains, the same recipe should transfer to other medical imaging tasks and protected attributes such as sex or ethnicity in radiology.","The sensitivity to domain order implies the method's fairness behavior is schedule-dependent, so real deployments would need to fix and document the domain ordering.","The FAT(E) trade-off scores are relative to a specific baseline model, so they are not comparable across papers unless the same vanilla backbone and denominator are used."],"forward_implications":["On Fitzpatrick-17k, FairDD lowers EOpp0 from 1.00 to 0.48 and EOdd from 10.54 to 5.65 while keeping accuracy at 86.53 versus 87.53 for vanilla, a drop of about one point.","On ISIC 2019, FairDD reports EOpp0 0.59, EOpp1 4.58, and EOdd 4.21, improving fairness over vanilla while staying within roughly two accuracy points of it.","Training order matters: the dark-to-light domain order improves both accuracy and fairness, while light-to-dark reverses the benefit, so the paper's results are tied to that schedule.","Ablations show the best trade-off occurs at distillation weight $\\alpha=0.6$ and that fairness first improves then declines as the statistical-parity weight $\\beta$ increases.","Larger replay buffers improve EOpp1 and EOdd with diminishing returns after capacity around 300."],"supporting_citations":[{"why":"Supplies the Fitzpatrick-17k dataset with skin-type annotations that define the sensitive-attribute domains.","marker":"[20]"},{"why":"Supplies the ISIC 2019 dataset, where age is the sensitive attribute used in the fairness evaluation.","marker":"[21]"},{"why":"FairAdaBN is the strongest fairness baseline; FairDD's trade-off improvement is measured against it.","marker":"[7]"},{"why":"Provides the mixup interpolation used by the cross-domain mixup module.","marker":"[13]"},{"why":"Defines the supervised contrastive loss that pulls same-class embeddings together.","marker":"[14]"},{"why":"Gives the statistical parity disparity definition formalized as Eq. (3).","marker":"[15]"},{"why":"Defines equal opportunity and equal odds, the fairness metrics reported in Table I.","marker":"[22]"},{"why":"Supplies the balanced fine-tuning idea behind the distillation fine-tuning module.","marker":"[19]"},{"why":"Supports the memory replay strategy used to construct the bounded replay buffer.","marker":"[12]"},{"why":"Provides resampling and independent baselines that FairDD is compared against in Table I.","marker":"[8]"}],"fun_headline_variants":["FairDD: incremental domains make skin-lesion AI fairer","FairDD: fairer skin diagnosis with domain replay","FairDD: balancing accuracy and fairness in derm AI","FairDD: fair derm diagnosis, minimal accuracy hit","FairDD: domain-incremental learning for fair derm AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The fairness gains rest on the statistical-parity disparity term in Eq. (3) being an optimizable training objective, but as printed it averages hard indicators of predicted labels, which give the network no gradient to update from.","fun_headline_variants_meta":{"raw":{"variants":["FairDD: incremental domains make skin-lesion AI fairer","FairDD: fairer skin diagnosis with domain replay","FairDD: balancing accuracy and fairness in derm AI","FairDD: fair derm diagnosis, minimal accuracy hit","FairDD: domain-incremental learning for fair derm AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000566,"raw_usage":{"total_tokens":2656,"prompt_tokens":893,"completion_tokens":1763,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":1680}},"tokens_in":509,"tokens_out":1763,"duration_ms":11961,"temperature":1.0,"reasoning_tokens":1680,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:29:01.062495+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Implement Eq. (3) literally during training: each term contains an expectation over $\\mathbb{1}\\{q(X_i)=y\\}$, a step function whose gradient is zero almost everywhere, so backpropagation cannot change the network through this loss. If the reported fairness gains persist with this literal implementation, the gains come from the other components; if they disappear, the statistical-parity term as written is not the mechanism, and the paper's stated objective must be a different, unstated function.","supporting_citations":[{"cited_title":"Fairness in cardiac mr image analysis: an investigation of bias due to data imbalance in deep learning based segmentation,","cited_arxiv_id":null,"evidence_quote":"Provides resampling and independent baselines that FairDD is compared against in Table I."},{"cited_title":"Towards transparency in dermatology image datasets with skin tone annotations by experts, crowds, and an algorithm,","cited_arxiv_id":null,"evidence_quote":"Supplies the Fitzpatrick-17k dataset with skin-type annotations that define the sensitive-attribute domains."},{"cited_title":"Fairadabn: Miti- gating unfairness with adaptive batch normalization and its application to dermatological disease classification,","cited_arxiv_id":null,"evidence_quote":"FairAdaBN is the strongest fairness baseline; FairDD's trade-off improvement is measured against it."},{"cited_title":"Pcr: Proxy-based contrastive replay for online class-incremental continual learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the balanced fine-tuning idea behind the distillation fine-tuning module."},{"cited_title":"Class-incremental continual learning into the extended der-verse,","cited_arxiv_id":null,"evidence_quote":"Supports the memory replay strategy used to construct the bounded replay buffer."}],"review_version":1}