Fairness at Every Intersection: Uncovering and Mitigating Intersectional Biases in Multimodal Clinical Predictions

Ayaazuddin Mohammad; Kishore Sampath; Malihe Alikhani; Resmi Ramachandranpillai

REVIEW 5 major objections 6 minor 1 cited by

Fairness at Every Intersection: Uncovering and Mitigating Intersectional Biases in Multimodal Clinical Predictions

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A subgroup-specific ensemble makes multimodal clinical predictions fairer at demographic intersections.

desk verdict SDAE is a plausible new idea for intersectional fairness in multimodal clinical models, but the paper overclaims robustness across embeddings and the evaluation lacks error bars. read the letter →

arxiv 2412.00606 v1 pith:6W6XMPWW submitted 2024-11-30 cs.AI

Resmi Ramachandranpillai , Kishore Sampath , Ayaazuddin Mohammad , Malihe Alikhani This is my paper · ORCID

classification cs.AI

keywords intersectionalfairnessmultimodalclinicalpredictionbiasmitigationsubgroup-specificensembledemographicparitytruepositiverateworst-caselanguagemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that fairness interventions aimed at a single demographic attribute, such as race alone or gender alone, can miss or even worsen bias at the intersections of those attributes, and that such intersectional bias is especially unstable when predictions draw on multiple data modalities. It introduces Subgroup-specific Discrimination Aware Ensembling (SDAE), a post-processing method that trains subgroup-pair-specific multimodal classifiers and combines them in a discrimination-aware vote. On two emergency-department datasets, MIMIC-Eye and MIMIC-IV ED, the authors report that SDAE improves demographic parity, true-positive-rate parity, and worst-case parity for demographic subgroups while keeping predictive performance nearly unchanged. The stakes are practical: if the method holds, clinical prediction models could be audited and corrected at the subgroup level rather than at the level of single protected attributes.

What carries the argument

The load-bearing mechanism is SDAE: every modality, including structured data, clinical notes, labs, events, and X-ray reports, is first converted into a unified text representation; frozen clinical BERT-family encoders, MedBERT, ClinicalBERT, and Bio-ClinicalBERT, embed that text; bootstrap splits of the intersectional subgroups train pairwise subgroup-specific classifiers; at inference those classifiers vote, and when they disagree a weighted score $\eta$ with a subgroup-specific threshold $\tau_{SG}$ decides the final label. A second piece is the worst-case parity metric, the min-max ratio of fairness values across subgroups, which is how the paper measures intersectional demographic parity and true-positive-rate parity. This design targets the disagreement region of subgroup-specific models, which is exactly where single-attribute post-processing fails.

What would settle it

Apply SDAE to a naturally skewed clinical dataset, or to a synthetic sample with realistic correlation structure, and check whether subgroup demographic parity and true-positive-rate parity improve for every minority subgroup while no subgroup drops more than 5% relative to the base model; a single subgroup whose fairness worsens under SDAE would falsify the robustness claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that intersectional bias in multimodal clinical predictions can be mitigated by a subgroup-specific ensemble rather than by single-attribute post-processing. On MIMIC-Eye and on a deliberately biased sample of MIMIC-IV ED, the base model showed hidden disparities at race-gender intersections even when single-attribute fairness looked acceptable; applying Reject Option Classification on gender, race, or both either injected new bias into correlated subgroups or produced no intersectional gain. SDAE, by contrast, consistently improves subgroup demographic parity, true-positive-rate parity, and worst-case parity, and specifically raises minority subgroup performance, for example Asian demographic parity in ED disposition from 0.575 to 0.601, instead of leveling all groups down. The authors conclude that SDAE is stable across datasets, subgroups, and clinical language model embeddings, with negligible performance degradation.

Load-bearing premise

The load-bearing premise is that the deliberately biased MIMIC-IV ED sample behaves like real-world intersectional disparities, so the robustness measured on it will transfer to naturally occurring data.

Editorial extensions

If this is right

Subgroup-level evaluation should become the default: single-attribute demographic parity and true-positive-rate tables can look fair while race-gender intersections hide severe disparities, as with Asian Female patients in ED disposition.
Post-processing mitigation should be checked for leveling-down: Reject Option Classification raised worst-case parity in some settings while dropping Asian demographic parity from 0.575 to 0.468, so fairness metrics alone can mask group-wise harm.
Unified textual representations plus frozen clinical language models allow small multimodal datasets to support downstream prediction and fairness analysis without training from scratch.
SDAE improves minority demographic parity and true-positive-rate parity on both a naturally structured multimodal dataset and a deliberately biased sample, supporting the claim that subgroup-specific ensembling generalizes across data distributions.
Choosing the clinical language model embedding matters for the fairness-accuracy trade-off: MedBERT gave the best F1 in the reported tasks, and the paper evaluates fairness across all three embeddings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the authors leave implicit is that SDAE operates only on predictions, so it could be dropped onto any multimodal classifier, not just text-unified clinical models, as long as subgroup-specific votes can be computed; this portability is testable outside electronic health records.
The robustness result leans on how realistic the synthetic MIMIC-IV ED biased sample is; a sharper stress test would apply SDAE to naturally occurring subgroup skew rather than an injected, cleanly separable bias sample.
The paper's own observation that worst-case parity ratios can hide group-wise drops suggests a better fairness metric would report both the min-max ratio and the absolute minority-group value; developing such a metric is a direct next step.
For high-cardinality sensitive attributes, SDAE's combinatorial growth in subgroup-pair models may need approximation, for example clustering subgroups, before clinical deployment, since the paper flags computational expense as a limitation.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper's core move—training subgroup-specific multimodal classifiers and ensembling them in the disagreement region—is a legitimate new variation for intersectional fairness. The empirical finding that single-attribute ROC can worsen disparities at intersections is useful and deserves attention. But the headline claim of robustness across embeddings is not supported: SDAE is only evaluated with Med-BERT. Tables 2 and 3 report single-point values with no confidence intervals, despite the reproducibility checklist claiming statistical tests. The biased MIMIC-IV ED sample is referenced only to a supplementary section that is not included. These gaps matter because they sit on the central claim.

What the paper does well: it takes a real problem—intersectional bias in multimodal clinical predictions—and proposes a post-processing method that is transparent about its scope. The analysis of how ROC on gender or race injects bias into intersectional subgroups is a nice concrete demonstration. The authors also acknowledge computational cost and the post-process limitation, and the ethics statement is sensible.

The soft spots, in proportion: (1) The 'across embeddings' claim. Table 1 benchmarks three embeddings for base classifiers, but SDAE appears only with Med-BERT. Either report SDAE with the other embeddings or drop 'embeddings' from the robustness claim. (2) No error bars or significance tests, contrary to the checklist. This is fixable but essential for a fairness claim. (3) The biased sample construction is undisclosed. If the bias injection creates cleanly separable clusters, SDAE's gains may not generalize. (4) The conclusion says 'without group-wise reduction in DP,' but Table 2 shows small reductions for White and Black subgroups; they are under 5%, but the wording is stronger than the data. (5) The ensembling rule in Equation 11 leaves h underspecified; a reader cannot reproduce the voting scheme.

On balance, the idea is worth a referee's time. The paper is not there yet, but the gaps are fixable, and the core direction is sound. Send it to review with a request for major revision, not a desk reject.

Referee Report

5 major / 6 minor

Summary. The paper proposes Subgroup-specific Discrimination Aware Ensembling (SDAE), a post-processing ensemble method for mitigating intersectional biases in multimodal clinical predictions. The authors convert all EHR modalities (notes, X-rays, events, labs, structured data) into a unified textual representation, embed it with clinical language models, and train subgroup-specific classifiers whose disagreement region is exploited to adjust predictions toward fairness. They evaluate on MIMIC-Eye and a biased sample of MIMIC-IV ED, comparing SDAE against reject option classification (ROC) applied to gender, race, and their intersection. The central claim, stated in the abstract and Section 4.2, is that SDAE is robust across datasets, subgroups, and embeddings, with consistent fairness improvements and negligible performance loss.

Significance. Intersectional fairness in multimodal clinical data is an important and understudied problem, and the paper addresses it with a conceptually simple post-processing approach that is more flexible than in-processing or pre-processing alternatives. The unified textual representation of multimodal EHR is a reasonable design choice, and the paper provides a useful empirical comparison with single-attribute ROC mitigation. The authors are explicit about some limitations, such as computational cost for many intersections, and include an ethics statement. However, the significance is diminished by the gap between the stated claims and the reported experiments: SDAE is only evaluated with Med-BERT, no statistical uncertainty is reported despite the checklist asserting otherwise, and the results in the paper's own tables do not consistently support 'consistent improvement' or 'no group-wise reduction.' The manuscript would be a valuable contribution if these gaps are addressed by additional experiments, uncertainty quantification, and corrected or softened claims.

major comments (5)

[Abstract and Section 4.2] The abstract and Section 4.2 claim that SDAE is 'robust across different datasets, subgroups, and embeddings,' but SDAE results are reported only for Med-BERT. Table 1 compares base classifiers with Med-BERT, Clinical BERT, and BioClinical BERT, but Tables 2 and 3 and Figures 3 and 4 contain no SDAE results for the latter two embeddings. The text in Section 4.1 even says 'In this work, we perform analysis using Med-BERT,' which explicitly narrows the scope. The 'across embeddings' component of the central claim is therefore unsupported by the reported experiments.
[Tables 2 and 3] The claim in Section 4.2 that SDAE shows 'consistent improvement in fairness (DP, TPR, and WP) across all tasks and datasets' is contradicted by the paper's own tables. In Table 2, SDAE's DP for White-M, White-F, Black-M, Black-F, and Asian-F is lower than the corresponding Mbase values (e.g., White-F drops from 0.719 to 0.709, Asian-F from 0.511 to 0.510), and TPR for Black-F decreases from 0.997 to 0.996. The conclusion that SDAE achieves fairness 'without reduction in group-wise DP' is not supported. Similar DP decreases appear in Table 3 for White-M and White-F. The manuscript needs to either revise these claims or show that the differences are within noise, which requires uncertainty estimates.
[Reproducibility Checklist, items 20 and 21] The reproducibility checklist states that the paper reports measures of variation and statistical significance tests, but Tables 2 and 3 present single-point DP and TPR values with no confidence intervals, standard deviations, or test results. Figures 3 and 4 also appear to lack error bars. Given the small sizes of some subgroups (e.g., Asian patients), the claimed improvements cannot be distinguished from random fluctuation. The authors should provide the variance information promised in the checklist or remove those checklist affirmations.
[Section 1 and the MIMIC-IV ED biased sample] The robustness claim across datasets relies heavily on the biased sample created from MIMIC-IV ED, but its construction is only described as being in 'supplementary file, Section 5,' which is not included in this arXiv submission. Without knowing how the bias was injected, it is impossible to assess whether the dataset reflects realistic intersectional disparities or artificially separable subgroup distributions. This directly bears on the 'robust across datasets' claim and should be moved into the main text or an available appendix.
[Table 2 caption and Section 4.2] The caption of Table 2 states that ROC applied to gender, race, and gender+race 'obtained identical results for all three ROC models.' This is surprising and unexplained: reject option classification with different sensitive attributes would normally produce different predictions. If the implementation collapses to a single model, then the conclusion that existing ROC methods cannot handle intersectionality is weakened. The authors should clarify the implementation and explain why all three variants lead to identical results.

minor comments (6)

[Throughout] The manuscript repeatedly refers to 'supplementary file, Section 1-6' for dataset statistics, experimental settings, additional results, and the biased-sample details, but no supplementary material is included with this arXiv submission. This hampers reproducibility and should be fixed by including the appendix or a pointer to a public repository.
[Equation (10)] Equation (10) writes the classifier output as 'min_loss_task(Classifier(...))', which is not a standard or clear description of training versus inference. Please separate the prediction function from the loss minimization used during training.
[Section 2] The notation 'SG_{gs1×gs2×...gsm}' is ambiguous; the set of intersectional subgroups should be defined more formally, and the indexing of gs values should be specified.
[Definitions 2] Definition 2 is labeled 'Equal Opportunity or True Positive Rate Parity' and the formula is correct for TPR parity, but the surrounding text should consistently use one term to avoid confusion with the demographic parity definition.
[Table 1 and Section 3.2] The paper alternates between 'Clinical BERT,' 'BioClinical BERT,' and 'Clinical BioBERT' when referring to the same embeddings; please unify the naming.
[Equation (11) and Section 4.2] The threshold tau_SG is fixed at 0.5, yet the text calls it a hyperparameter for balancing fairness and accuracy. No sensitivity analysis or selection criterion for tau_SG is provided, so the robustness of the fairness results with respect to this threshold is unclear.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SDAE's fairness improvements are empirical, evaluated on held-out predictions with fixed thresholds and external baselines.

full rationale

The paper's central claim is empirical: SDAE improves intersectional fairness (DP, TPR, WP) relative to base and ROC baselines. The derivation chain does not reduce to its inputs. Fairness metrics are computed on test/validation predictions after training; the ensemble threshold tau_SG is fixed at 0.5 (Section 3.3, Eq. 11) and the mixing weight h is derived from the number of models, not tuned to maximize reported fairness. The subgroup-specific models are trained on subgroup-pair splits, but nothing in Eq. 11 enforces the reported DP/TPR values; the improvement is an empirical outcome, not a fitted parameter renamed as a prediction. The baseline comparisons to ROC (Kamiran et al. 2012) are external, and the fairness definitions come from Ghosh et al. (2021), cited independently. The reference list contains no self-citations, and no uniqueness theorem is imported from the authors' prior work. The paper does contain a non-circular evidence gap: the abstract claims SDAE is robust across embeddings, yet SDAE experiments (Tables 2, 3) use only Med-BERT, with Clinical BERT and BioClinical BERT appearing only in the base-model modality analysis (Table 1). This is an unsupported generalization claim, not a circular definition or fitted-input-as-prediction, so it does not affect the circularity score. The Limitations section acknowledges computational cost and post-process-only scope; these are limitations, not circularity.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The framework rests on several domain assumptions about data conversion, sample representativeness, metric choice, and the value of disagreement ensembling. The only hand-tuned free parameter reported is the threshold tau. No new physical or conceptual entities are introduced.

free parameters (1)

tau (threshold) = 0.5 (default; noted as tunable per subgroup)
Eq. 11 in Section 3.3 defines tau_SG as a threshold for classifying instances in the disagreement region. The paper sets it to 0.5 for all experiments and says it can be adjusted to balance fairness-accuracy. No sensitivity analysis is reported, so the fairness results depend on this hand-chosen value.

assumptions (5)

domain assumption Representing all modalities as unified text preserves sufficient clinical information for the downstream prediction tasks.
Section 3.1 converts X-rays, events, labs, and structured data into text. If this conversion loses modality-specific signal, the reported predictions and fairness metrics may not reflect true multimodal performance.
domain assumption The hand-crafted biased sample from MIMIC-IV ED is representative of real-world intersectional disparities.
Section 4.1 states MIMIC-IV ED lacks severe disparities, so a biased sample was created. The construction is only referenced to supplementary Section 5, leaving the dataset's validity and generalizability unverifiable.
standard math Worst-case parity with the 80% rule is an appropriate measure of intersectional fairness.
Equations (4) and (5) adopt WP from Ghosh et al. 2021, and the interpretation relies on the 80% rule, a convention not derived in the paper.
domain assumption Pre-trained clinical language models generate meaningful embeddings from the unified text even on small datasets.
Section 3.2 relies on Med-BERT, ClinicalBERT, and BioClinicalBERT to produce feature vectors. The paper does not validate the quality of these embeddings for the new textualized modalities.
domain assumption The disagreement region of subgroup-specific classifiers can be exploited to improve fairness without significant accuracy loss.
This is the core hypothesis behind SDAE in Section 3.3, but it is only empirically asserted. No theoretical justification or ablation is provided to isolate why disagreement helps.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Fairness at Every Intersection: Uncovering and Mitigating Intersectional Biases in Multimodal Clinical Predictions." pith.science (2026). https://pith.science/paper/6W6XMPWW

@misc{pith2026241200606,
  author       = {Pith},
  title        = {Pith review of: Fairness at Every Intersection: Uncovering and Mitigating Intersectional Biases in Multimodal Clinical Predictions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6W6XMPWW}},
  note         = {Machine review of arXiv:2412.00606}
}

read the original abstract

Biases in automated clinical decision-making using Electronic Healthcare Records (EHR) impose significant disparities in patient care and treatment outcomes. Conventional approaches have primarily focused on bias mitigation strategies stemming from single attributes, overlooking intersectional subgroups -- groups formed across various demographic intersections (such as race, gender, ethnicity, etc.). Rendering single-attribute mitigation strategies to intersectional subgroups becomes statistically irrelevant due to the varying distribution and bias patterns across these subgroups. The multimodal nature of EHR -- data from various sources such as combinations of text, time series, tabular, events, and images -- adds another layer of complexity as the influence on minority groups may fluctuate across modalities. In this paper, we take the initial steps to uncover potential intersectional biases in predictions by sourcing extensive multimodal datasets, MIMIC-Eye1 and MIMIC-IV ED, and propose mitigation at the intersectional subgroup level. We perform and benchmark downstream tasks and bias evaluation on the datasets by learning a unified text representation from multimodal sources, harnessing the enormous capabilities of the pre-trained clinical Language Models (LM), MedBERT, Clinical BERT, and Clinical BioBERT. Our findings indicate that the proposed sub-group-specific bias mitigation is robust across different datasets, subgroups, and embeddings, demonstrating effectiveness in addressing intersectional biases in multimodal settings.

Figures

Figures reproduced from arXiv: 2412.00606 by the authors.

**Figure 2.** The proposed architecture We use F1 (Hicks et al. 2022) to compare the performances of these tasks. The experimental settings are given in the supplementary file, Section 3. Multimodality selection: We analyze the model’s performance change when supplemented with modality across embeddings. The underlying idea is that a well-designed model trained from multiple sources should not exhibit performance degradation (C… view at source ↗

**Figure 3.** Comparison of WP in multitask setting using MIMIC-Eye [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Comparison of F1 scores [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

The Multimodal Paradox: How Added and Missing Modalities Shape Bias and Performance in Multimodal AI
cs.AI 2025-05 conditional novelty 4.0 of 10

On two emergency-department datasets, adding modalities during training improved classification performance while fairness changes were inconsistent, and masking modalities at inference degraded both performance and fairness.

Reference graph

Works this paper leans on

23 extracted references · 13 canonical work pages · cited by 1 Pith paper

[1]

Includes a conceptual outline and/or pseudocode descrip- tion of AI methods introduced (yes)
[2]

Clearly delineates statements that are opinions, hypothe- sis, and speculation from objective facts and results (yes)
[3]

Provides well marked pedagogical references for less- familiare readers to gain background necessary to repli- cate the paper (yes)

work page
[4]

Does this paper make theoretical contributions? (no)

work page
[5]

A motivation is given for why the experiments are con- ducted on the selected datasets (yes)

work page
[6]

(yes/links are given)

All novel datasets introduced in this paper are included in a data appendix. (yes/links are given)

work page
[7]

All novel datasets introduced in this paper will be made publicly available upon publication of the paper with a license that allows free usage for research purposes. (NA)
[8]

All datasets drawn from the existing literature (potentially including authors’ own previously published work) are accompanied by appropriate citations. (yes)

work page

Show all 23 references

[9]

(partial/have to complete training)

All datasets drawn from the existing literature (potentially including authors’ own previously published work) are publicly available. (partial/have to complete training)
[10]

All datasets that are not publicly available are described in detail, with explanation why publicly available alterna- tives are not scientifically satisficing. (yes)
[11]

Does this paper include computational experiments? (yes) If yes, please complete the list below
[12]

Any code required for pre-processing data is included in the appendix. (yes)
[13]

All source code required for conducting and analyzing the experiments is included in a code appendix. (yes)
[14]

All source code required for conducting and analyzing the experiments will be made publicly available upon publication of the paper with a license that allows free usage for research purposes. (yes)
[15]

All source code implementing new methods have com- ments detailing the implementation, with references to the paper where each step comes from (yes)
[16]

If an algorithm depends on randomness, then the method used for setting seeds is described in a way sufficient to allow replication of results. (yes)
[17]

This paper specifies the computing infrastructure used for running experiments (hardware and software), including GPU/CPU models; amount of memory; operating system; names and versions of relevant software libraries and frameworks. (yes)
[18]

This paper formally describes evaluation metrics used and explains the motivation for choosing these metrics. (yes)
[19]

This paper states the number of algorithm runs used to compute each reported result. (yes)
[20]

Analysis of experiments goes beyond single-dimensional summaries of performance (e.g., average; median) to in- clude measures of variation, confidence, or other distribu- tional information. (yes)
[21]

The significance of any improvement or decrease in per- formance is judged using appropriate statistical tests (e.g., Wilcoxon signed-rank). (yes)
[22]

This paper lists all final (hyper-)parameters used for each model/algorithm in the paper’s experiments. (yes)
[23]

This paper states the number and range of values tried per (hyper-) parameter during development of the paper, along with the criterion used for selecting the final parameter setting. (yes)

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Includes a conceptual outline and/or pseudocode descrip- tion of AI methods introduced (yes)

[2] [2]

Clearly delineates statements that are opinions, hypothe- sis, and speculation from objective facts and results (yes)

[3] [3]

Provides well marked pedagogical references for less- familiare readers to gain background necessary to repli- cate the paper (yes)

work page

[4] [4]

Does this paper make theoretical contributions? (no)

work page

[5] [5]

A motivation is given for why the experiments are con- ducted on the selected datasets (yes)

work page

[6] [6]

(yes/links are given)

All novel datasets introduced in this paper are included in a data appendix. (yes/links are given)

work page

[7] [7]

All novel datasets introduced in this paper will be made publicly available upon publication of the paper with a license that allows free usage for research purposes. (NA)

[8] [8]

All datasets drawn from the existing literature (potentially including authors’ own previously published work) are accompanied by appropriate citations. (yes)

work page