{"id":"f9e25e62-108c-487c-91ea-18d6f8da1d54","arxiv_id":"2506.17787","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"FairMoE, a layer-wise mixture-of-experts model with mutual-information-based group specialization and soft routing, improves accuracy on Fitzpatrick-17k and ISIC 2019 while preserving equalized-odds fairness.","lead":"This paper introduces FairMoE, a mixture-of-experts network whose experts specialize by skin tone or age group, with a router that soft-assigns each image to experts. On two public skin disease datasets it reports higher diagnostic accuracy and equal or better fairness than several prior bias-mitigation methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Router group-prediction accuracy is unverified; if it is near chance, experts are not group-specialized and FairMoE's gains may stem from added MoE capacity rather than the proposed mechanism.","rationale":"The reader's weakest assumption identifies the router's confidence score as the load-bearing premise, and I agree. The paper's entire framing—group-specific experts, soft routing for boundary samples, and 'incorporating rather than eliminating'—relies on the router correctly estimating the sensitive attribute from images. If the router is inaccurate, the model is effectively a generic MoE with extra parameters, and the empirical gains could be a capacity effect. The paper's Figure 2a provides suggestive evidence that routing scores differ by skin type, but it does not quantify router accuracy or the achieved expert-group mutual information. This is a concrete, verifiable gap. The capacity-matched-baseline concern is related but secondary: even with a perfect router, the paper must show the group-specific mechanism, not just added parameters, drives the improvement. However, the router accuracy check is the most direct test of the proposed mechanism and would also inform the capacity interpretation. The conditional verdict is appropriate: the paper is promising but needs this verification before the central claim can be accepted. No new concern changes the verdict, so UNCHANGED is correct.","tokens_in":8502,"tokens_out":6332,"duration_ms":65228,"concrete_test":"On held-out test splits of Fitzpatrick-17k and ISIC 2019, compute (i) the router's accuracy and AUC for predicting the true sensitive attribute, and (ii) the empirical mutual information I(C; E_test) between true group labels and the argmax expert selected by the router at inference. If the AUC is near 0.5 or I(C; E_test) is close to 0, the experts are not group-specialized, and the accuracy gains should be attributed to the additional parameters rather than to the proposed mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism of FairMoE depends on the router's confidence score s_k(x) in Eq. (2) (Section 3.4) being an accurate estimate of the true sensitive attribute and of the sample's proximity to each group's distribution. The mutual-information loss in Section 3.3 can enforce specialization on the training set, but if the router does not generalize to test samples, the experts will not be group-specialized at inference time, and the soft routing described as the key to handling boundary samples collapses. The paper never reports router accuracy, AUC, calibration, or the empirical mutual information between true group labels and the selected experts on held-out data. Given that the layer-wise MoE adds a second expert per layer—roughly doubling the parameters of the backbone—the observed accuracy improvements over the baselines could simply reflect increased capacity rather than the proposed group-specific routing. Without evidence that the router actually infers skin type or age, the central claim that FairMoE improves accuracy by 'incorporating rather than eliminating' sensitive attributes is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FairMoE, a layer-wise mixture-of-experts (MoE) architecture for skin disease diagnosis in which each expert is encouraged, via a mutual-information loss, to specialize on one sensitive-attribute group (skin type or age), while a soft probabilistic router uses the router's confidence score to weight training data from other groups. On Fitzpatrick-17k and ISIC 2019, the authors report higher F1 scores and lower equalized-odds values than several fairness baselines, arguing that incorporating sensitive attributes improves accuracy without sacrificing fairness, in contrast to fairness-through-invariance methods.","tokens_in":8729,"tokens_out":4779,"duration_ms":49738,"significance":"If the results hold, FairMoE offers a useful counterpoint to the dominant fairness-through-invariance paradigm in medical imaging: rather than removing sensitive attributes, it exploits them through group-specialized experts and soft routing. The framework is conceptually simple, builds on a standard MoE architecture, and is evaluated on two public benchmarks against recent pruning and quantization baselines. The paper includes an ablation over the number of MoE layers and a case study showing routing-score behavior across layers, which are helpful. However, the strength of the empirical claim depends on several unverified assumptions: router reliability, capacity matching, and statistical stability. The current manuscript does not yet provide enough evidence to fully support the claim of simultaneous accuracy gain and fairness preservation.","major_comments":[{"comment":"The central empirical claim of 'substantial accuracy improvements' is not supported by statistical evidence. No error bars, confidence intervals, or number of seeds are reported for any entry in Table 1. The F1 difference between FairMoE (0.502) and SCP-FairPrune (0.476) on Fitzpatrick-17k is 0.026, and the Eodd differences are even smaller (0.154 vs. 0.164), which may be within run-to-run variation. Please report mean and standard deviation over at least three independent runs and, where possible, a paired significance test for the accuracy and fairness metrics.","section":"Section 4, Table 1"},{"comment":"FairMoE roughly doubles the number of parameters in each converted layer by introducing two experts per layer, while the baselines (Vanilla, FairAdaBN, SCP-FairPrune, FairQuantize) use a single weight set. The reported accuracy gains could therefore be partly attributable to added model capacity rather than to the proposed group-specialized routing mechanism. Please include a capacity-matched baseline, for example a standard MoE without the mutual-information loss or a single wide network with a comparable parameter count, so that the effect of the group-specialization mechanism is isolated.","section":"Section 4, Table 1; Section 3.2"},{"comment":"The mechanism assumes that the router's confidence score s_k(x) is a reliable estimate of the true sensitive attribute and of the sample's proximity to each group's distribution, yet the paper does not report router accuracy, AUC, calibration, or the empirical mutual information between true group labels and selected experts on held-out data. Fig. 2(a) shows average routing scores across layers, but not prediction accuracy. For ISIC 2019, where the sensitive attribute is age (<=55 vs. >55), it is especially unclear whether the router can infer age from dermoscopic images. If the router is near chance, expert specialization may not occur at inference, and the claimed benefit of soft routing collapses. Please quantify router performance and, if necessary, analyze failure cases.","section":"Section 3.4, Eq. (2); Section 4, Fig. 2"},{"comment":"The methodology for the mutual-information loss is underspecified. The text defines P(E_i|C_j) but Eq. (1) uses the joint P(C_i,E_j), and it is unclear how these probabilities are estimated (per batch or over the whole dataset) and how the loss is differentiated through the router's discrete assignments. In addition, Eq. (2) defines P(E_k|x) with alpha_k = 1/N_k; if s_k(x) is a softmax posterior probability, multiplying by 1/N_k changes the normalization in a way that is not justified, and it is unclear whether alpha is applied to logits or probabilities. Please clarify the notation and the exact computation.","section":"Section 3.3, Eq. (1); Section 3.4, Eq. (2)"}],"minor_comments":[{"comment":"The phrase 'the lost of clinically relevant diagnostic cues' should be 'the loss of clinically relevant diagnostic cues'.","section":"Abstract"},{"comment":"The statement that 'malignant melanoma is 20 times more common in African Americans compared to other groups [16]' appears inconsistent with the cited source and with the general epidemiology of melanoma; please verify the direction of this claim.","section":"Section 2"},{"comment":"The metrics Eopp0 and Eopp1 are used in Table 1 but are never defined; please define these subscripts (for example, whether they refer to specific classes or groups) and how they are computed.","section":"Section 4, Fairness Metrics"},{"comment":"The caption should define 'Avg.' and 'Diff.'; in particular, 'Diff.' appears to be the absolute difference between groups, but this should be stated explicitly.","section":"Table 1"},{"comment":"For the Vanilla baseline, all FATE values are 0.000. Please explain how FATE is normalized and why Vanilla is defined as zero; otherwise the FATE comparisons are difficult to interpret.","section":"Table 1"},{"comment":"The axes are unclear: F1 Score and Eodd appear to share the same y-axis label but have different scales. Please use separate axes or a clear legend.","section":"Fig. 2(b)"},{"comment":"There is a typo: 'group-specific modules for fairnessis' should be 'group-specific modules for fairness'.","section":"Section 4, ISIC 2019 paragraph"},{"comment":"The phrase 'weighted by a parameterwM I' has a missing space and should read 'weighted by a parameter w_MI'.","section":"Section 3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is appropriate for a medical imaging audience, and the core idea is interesting. The main empirical claim is currently under-supported because of the missing capacity-matched baseline, missing router diagnostics, and lack of error bars; these issues are fixable within the scope of a revision. I would not recommend acceptance until those points are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid MICCAI-level idea with a real experimental gap. The authors propose FairMoE, a layer-wise MoE where each expert is nudged toward a demographic group via a mutual-information loss, then use soft probabilistic routing to handle boundary samples. On Fitzpatrick-17k and ISIC 2019, they report higher F1 than several fairness baselines while keeping equalized odds comparable. That's a genuinely interesting result, and the motivation—removing sensitive attributes discards clinically relevant cues like skin type—is well argued.\n\nWhat's new: the combination of MI-based expert-group alignment plus soft routing is not in the cited fairness literature. The paper also gives a useful case study showing routing scores strengthen with depth, and an ablation on the number of MoE layers that suggests the mechanism helps.\n\nThe soft spots, in order. First, there is no capacity-matched baseline. Replacing each conv layer with two experts roughly doubles parameters; the accuracy jump over SCP-FairPrune and FairQuantize could simply reflect more capacity. A baseline with a wider backbone or a MoE with unconstrained routing (no MI loss) would settle this. Without it, the central claim 'we avoid the trade-off' is not established. Second, the router itself is never evaluated. The whole mechanism hinges on the router's score s_k(x) being an accurate estimate of the true group (skin type or age) and of proximity to each group's distribution. If the router is near chance on held-out data, the experts aren't group-specialized at inference, and the boundary-sample argument collapses. Please report router accuracy or AUC, and the actual mutual information between true labels and selected experts on test data. Third, no error bars anywhere; for fairness metrics especially, standard deviations over runs matter. Fourth, the MI loss in Eq. (1) is described briefly; a batch-level estimate is not specified. This is minor.\n\nNone of these are fatal. The idea is sound, the datasets are public, and the consistent improvements across two datasets and several baselines earn real credit. But the paper overstates its conclusion. It should be cited for the approach, not as proof that incorporating sensitive attributes beats eliminating them.\n\nFor peer review: yes, send it out. A good referee would ask for the capacity control and router analysis, and the paper would come back stronger. For a reading group, it's a good discussion piece on fairness-capacity confounds.","headline":"FairMoE's group-specialized MoE is a promising way to use sensitive attributes in skin AI, but the missing capacity-matched baseline and unverified router mean the core claim isn't proven yet.","tokens_in":9211,"tokens_out":2666,"would_cite":true,"duration_ms":25899,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FairMoE turns sensitive attributes into group-specific experts, raising skin-diagnosis accuracy while keeping fairness.","keywords":["fairness","skin disease diagnosis","mixture-of-experts","group-specific experts","soft routing","mutual information","Fitzpatrick-17k","ISIC 2019"],"falsifier":"Measure the router's agreement with true skin-type or age labels on held-out test data; if that agreement is near chance while FairMoE still reproduces the reported F1 and equalized-odds numbers, then the group-specialization mechanism is not what produces the gains. Alternatively, setting the mutual-information loss term to zero and observing that the gains disappear would confirm the specialization term is doing the work.","tokens_in":8335,"feed_emoji":"🩺","tokens_out":7605,"duration_ms":68806,"temperature":0.7,"pith_summary":"The paper argues that the standard strategy of removing sensitive attributes from skin-disease classifiers is self-defeating because traits like skin type and age carry clinically relevant diagnostic signal. It proposes FairMoE, a convolutional network in which each convolutional layer is replaced by a mixture-of-experts module with one expert per demographic group, trained so that routers prefer the matching expert while still allowing soft assignment of borderline samples. On Fitzpatrick-17k and ISIC 2019, the authors report that FairMoE raises the average F1 score over the best fairness baselines (0.502 vs 0.476, and 0.767 vs 0.737) while keeping equalized-odds gaps at or below the baselines. The significance, if these results hold, is that fairness need not be purchased by lowering accuracy: incorporating group information can help all groups, especially the unprivileged one.","feed_headline":"FairMoE lifts accuracy without sacrificing fairness in skin diagnosis","feed_subtitle":"On Fitzpatrick-17k and ISIC 2019, group-specific experts beat fairness baselines on both accuracy and equalized odds.","key_machinery":"The load-bearing object is the layer-wise mixture-of-experts module: each convolutional layer of a ResNet-18 or VGG-11 backbone is replaced by several experts (one per group) and a small routing convolution. Two mechanisms make it work: a mutual-information loss $-I(C;E_Y)$ that establishes a strong correlation between group labels and expert choice, and a soft routing rule that weights the router's confidence score by inverse group size, so selection probability reflects a sample's proximity to each group rather than a hard label. The router's confidence score performs double duty: it is the training signal for specialization and the inference-time routing probability.","core_discovery":"The central claim is that fairness in skin-disease diagnosis can be improved by incorporating sensitive attributes rather than eliminating them, through a layer-wise mixture-of-experts architecture with group-specific experts. FairMoE makes the router specialize by maximizing the mutual information $I(C;E)$ between group labels and expert assignments, then uses a soft probabilistic routing rule $P(E_k|x) = \\alpha_k s_k(x) / \\sum_j \\alpha_j s_j(x)$, where $s_k(x)$ is the router's confidence that sample $x$ belongs to group $k$ and $\\alpha_k$ rebalances group sizes. This lets out-of-group data train an expert when the sample is close to that group's distribution, which is the proposed mechanism for avoiding the distribution shift of hard group-specific training and for handling boundary samples. In the reported experiments, FairMoE attains the highest average F1 on both datasets and equalized-odds values at or below all baselines, so the paper's claim is that accuracy and fairness improve together.","pith_inferences":["Editorial inference: if the router's confidence score is a good proxy for group proximity, the same architecture should transfer to other diagnostic imaging tasks with clinically relevant demographic attributes, such as age-defined subgroups in retinal or chest imaging.","Editorial inference: comparing FairMoE against a variant whose router is trained with explicit group labels would isolate whether the mutual-information objective or the soft routing itself drives the reported gains.","Editorial inference: treating skin tone as a continuous rather than binary grouping could make the soft-routing design exploit intermediate Fitzpatrick types instead of forcing them into two buckets."],"forward_implications":["Fairness methods that remove sensitive attributes are not the only viable route; incorporating those attributes can raise accuracy for both groups at once.","On the reported results, the unprivileged group improves more than the privileged group, so group-specific experts narrow the gap by lifting the weaker side rather than dropping the stronger side.","Because deeper MoE layers contribute more, practitioners with limited compute can replace only the deepest convolutional layers and retain most of the accuracy and fairness benefit.","Soft routing gives borderline samples access to more than one expert, which is the proposed answer to the hard-boundary problem of fixed group splitting."],"supporting_citations":[{"why":"Supplies the Fitzpatrick-17k dataset with skin-type labels used as the primary evaluation benchmark.","marker":"[10]"},{"why":"Supplies the ISIC 2019 dataset with age as the sensitive attribute, the second evaluation benchmark.","marker":"[6,21]"},{"why":"Introduces the sparsely-gated mixture-of-experts layer that FairMoE adapts for group-specific routing.","marker":"[19]"},{"why":"Provides robust mixture-of-expert training for convolutional neural networks, which FairMoE builds on.","marker":"[29]"},{"why":"FairPrune baseline whose training settings FairMoE reuses and whose accuracy and fairness numbers are compared.","marker":"[25]"},{"why":"SCP-FairPrune baseline, the best-performing previous fairness method on Fitzpatrick-17k in Table 1.","marker":"[15]"},{"why":"FairQuantize baseline, representing quantization-based fairness methods against which FairMoE is compared.","marker":"[11]"},{"why":"FairAdaBN baseline and the FATE metric used to evaluate the accuracy-fairness trade-off.","marker":"[26]"},{"why":"Evidence that bias grows in deeper layers, supporting the choice of layer-wise MoE placement.","marker":"[5]"},{"why":"DomainIndep baseline, the hard group-specific classifier approach that FairMoE improves upon.","marker":"[24]"}],"fun_headline_variants":["FairMoE: Fairness via incorporation, not elimination","FairMoE lifts accuracy and fairness in skin diagnosis","Group-specific experts improve skin diagnosis fairness and accuracy","Incorporating sensitive attributes improves skin AI fairness","FairMoE: Accuracy and fairness up in skin disease AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire mechanism rests on the router's confidence score being a good estimate of a sample's true group and of how close the sample is to each group's data distribution, and the paper reports no direct measurement of the router's accuracy.","fun_headline_variants_meta":{"raw":{"variants":["FairMoE: Fairness via incorporation, not elimination","FairMoE lifts accuracy and fairness in skin diagnosis","Group-specific experts improve skin diagnosis fairness and accuracy","Incorporating sensitive attributes improves skin AI fairness","FairMoE: Accuracy and fairness up in skin disease AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000707,"raw_usage":{"total_tokens":3163,"prompt_tokens":899,"completion_tokens":2264,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":2188}},"tokens_in":515,"tokens_out":2264,"duration_ms":14721,"temperature":1.0,"reasoning_tokens":2188,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:00:48.068827+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the router's agreement with true skin-type or age labels on held-out test data; if that agreement is near chance while FairMoE still reproduces the reported F1 and equalized-odds numbers, then the group-specialization mechanism is not what produces the gains. Alternatively, setting the mutual-information loss term to zero and observing that the gains disappear would confirm the specialization term is doing the work.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF International Conference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"Provides robust mixture-of-expert training for convolutional neural networks, which FairMoE builds on."},{"cited_title":"In: International Conference on Med- ical Image Computing and Computer-Assisted Intervention","cited_arxiv_id":null,"evidence_quote":"FairPrune baseline whose training settings FairMoE reuses and whose accuracy and fairness numbers are compared."},{"cited_title":"In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion","cited_arxiv_id":null,"evidence_quote":"SCP-FairPrune baseline, the best-performing previous fairness method on Fitzpatrick-17k in Table 1."},{"cited_title":"In: International Conference on Medical Image Computing and Computer-Assisted Intervention","cited_arxiv_id":null,"evidence_quote":"FairQuantize baseline, representing quantization-based fairness methods against which FairMoE is compared."},{"cited_title":"In: International Conference on Medical Image Computing and Computer-Assisted Intervention","cited_arxiv_id":null,"evidence_quote":"FairAdaBN baseline and the FATE metric used to evaluate the accuracy-fairness trade-off."},{"cited_title":"In: International Conference on Medical Image Computing and Computer-Assisted Intervention","cited_arxiv_id":null,"evidence_quote":"Evidence that bias grows in deeper layers, supporting the choice of layer-wise MoE placement."},{"cited_title":"In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","cited_arxiv_id":null,"evidence_quote":"DomainIndep baseline, the hard group-specific classifier approach that FairMoE improves upon."}],"review_version":1}