Pith. sign in

REVIEW 2 major objections 6 minor 63 references

Subgroups Matter for Robust Bias Mitigation

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Subgroup choice determines whether bias mitigation helps or hurts.

desk verdict Solid empirical case that subgroup choice can break bias mitigation; the KL diagnostic is a nice post-hoc story but not a general criterion. read the letter →

arxiv 2505.21363 v3 pith:7D3L4YHP submitted 2025-05-27 cs.LG cs.AI

Anissa Alloula , Charles Jones , Ben Glocker , Bart{l}omiej W. Papież This is my paper
classification cs.LGcs.AI
keywords subgroupdefinitionbiasmitigationspuriouscorrelationdistributionallyrobustoptimisationresamplingKLdivergencegroupfairnessgeneralisationunderdistributionshift
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to answer why bias mitigation methods so often fail to beat plain ERM, and its answer is that the choice of which subgroups to mitigate on does most of the work. Across four vision and language datasets and four mitigation methods, groupings built from where disparities are observed — for example the sex attribute S — frequently make test performance worse than no mitigation at all, while groupings built from the (A, Y) atom that carries the spurious correlation improve it substantially. The paper explains this with a diagnostic: a subgrouping is effective to the extent that reweighting its subgroups can bring the biased training distribution close to the unbiased test distribution, measured by the minimum achievable KL divergence between the two. If correct, this reframes subgroup definition from an annotation detail into the central lever for making bias mitigation robust, and it implies that fairness with respect to a given group may be best pursued with a different set of subgroups than the ones used to define that group.

What carries the argument

The machinery is the partition of the eight atomic (Y, S, A) cells into k subgroups together with the convex optimization $\min_{w \in \Delta^k} \mathrm{KL}(P^w_{\text{train}} \| P_{\text{unbiased}})$, where $P_{\text{unbiased}}$ is the uniform distribution over the eight cells and $P^w_{\text{train}}$ reweights each subgroup's atoms by the weights $w$. Pinsker's inequality converts this divergence into an upper bound on test error, test error $\le$ train error $+ \sqrt{\tfrac12 \mathrm{KL}}$, and because all trained models reach similarly low train error, the divergence term is the quantity that orders subgroupings. The calculation is done without training a model, so it acts as a pre-analysis of what any reweighting method can in principle recover; the paper verifies the ordering empirically across gDRO, resampling, DomainInd, and CFair.

What would settle it

One concrete check: in the same spurious-correlation setup, compare a low-KL subgrouping like (A, Y) with a high-KL one like S or (S, Y) using the same method, seeds, and hyperparameters; the KL account predicts the high-KL grouping never beats the low-KL one on unbiased test AUC. A single run where an S-based grouping outperforms (A, Y) — or a variant with, say, label noise where KL stays high but test AUC improves — would refute the mechanism.

Watch

Extended reading notes

Core claim

The central claim is that a subgrouping strategy succeeds in bias mitigation exactly when it allows the biased training distribution to be reweighted back toward the unbiased test distribution. Formally, the paper represents both distributions as probability vectors over the eight (Y, S, A) combinations and computes, for each candidate partition into subgroups, the minimum Kullback-Leibler divergence $\mathrm{KL}(P^w_{\text{train}} \| P_{\text{unbiased}})$ achievable by reweighting the subgroups (uniform weights for resampling, optimized weights for gDRO). This minimum divergence predicts unbiased test AUC with correlations between 0.73 and 0.97 across datasets, and it explains the otherwise paradoxical observations that S-based groupings hurt, that granularity and moderate annotation noise barely matter, and that the best grouping (Y, S, A) achieves divergence zero. The paper's counter-intuitive headline result follows from the same account: since the disparity across S is caused by the (A, Y) spurious correlation, mitigating on (A, Y) reduces the S disparity better — up to ten times smaller accuracy gaps — than mitigating on S itself, so the subgroups used to evaluate fairness need not be the subgroups used to achieve it.

Load-bearing premise

The load-bearing assumption is that the unbiased test distribution is known (here uniform over the eight (Y, S, A) combinations) and that everything that differs between training and testing shows up in the frequencies of those combinations; in real deployments the test distribution is unknown, so the diagnostic cannot be applied directly.

Editorial extensions

If this is right

  • gDRO and resampling improve unbiased test AUC only for subgroupings that lower the minimum achievable KL divergence (around 0.113 for $(A,Y)$ and $SC/no\text{-}SC$, and 0.000 for $(Y,S,A)$); groupings that leave it near 0.527 ($S$, $Y$, $A$ alone) match or underperform the ERM baseline.
  • Applying mitigation to the very subgroups where disparities are observed ($S$ and $(S,Y)$) is actively harmful in a majority of experiments — detrimental in 10 of 16 — so an observed disparity is not a license to mitigate on that grouping.
  • Fairness with respect to a set of subgroups can be improved by mitigating on a different set: $(A,Y)$ groupings deliver up to 0.21 higher worst-group accuracy across $S$ and up to ten times smaller accuracy gaps than $S$-based groupings.
  • Increasing subgroup granularity (e.g. $(A,Y)$ vs $(A,Y)_8$) and annotation noise below 10% barely change the achievable divergence or the results, so attribute choice matters far more than granularity or small label noise.
  • The same ordering holds for model-based methods (DomainInd, CFair): subgroups built on $A$ help (up to +0.07 AUC on CXP) while $S$ subgroups hurt (down to -0.14 for CFair on MNIST).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the KL diagnostic can be computed without any model training, so it could be used as a pre-screening test: practitioners with a candidate subgrouping and a guess at the deployment distribution could discard subgroups that cannot achieve a chosen divergence budget before spending compute.
  • Editorial inference: benchmark studies that compare mitigation methods should vary subgroup definitions and report the spread, since otherwise method rankings may reflect accidental subgroup choice rather than method quality.
  • Editorial inference: the results suggest decoupling fairness auditing from mitigation — audit on the socially salient groups, but select mitigation subgroups by what explains the error, since the two sets need not coincide.
  • Editorial inference: a direct testable extension is to apply the same divergence criterion to automatically discovered subgroups (inferred or clustered), checking whether a KL-minimizing partition coincides with discovered ones or merely with observed-disparity partitions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. This paper studies how the choice of subgroups used by bias mitigation methods affects their ability to improve generalisation on an unbiased test distribution. The authors construct four semi-synthetic datasets (MNIST, CheXpert, CelebA, CivilComments) with a spurious correlation between an attribute A and label Y that is distributed unevenly across a second attribute S, and train 306 models using gDRO, resampling, DomainInd, CFair, and JTT with a variety of subgroupings (coarse, fine-grained, intersectional, noisy, random). They report that subgroup choice strongly affects unbiased test AUC; in particular, subgroups based on S or (S,Y) are often worse than ERM, while subgroups involving (A,Y) improve performance. They propose a KL-divergence diagnostic: the minimum achievable KL divergence between a subgroup-weighted training distribution and the uniform unbiased test distribution correlates strongly with test AUC. They also report the counter-intuitive finding that disparities across S are best reduced by mitigating on (A,Y) subgroups rather than on S subgroups.

Significance. The paper addresses an understudied and practically important question: why bias mitigation methods often fail despite the use of reasonable-looking subgroups. The empirical contribution is substantial: 306 trained models with three seeds per setting, code release, and consistent trends across four datasets and four mitigation methods. The central empirical claim--that observing a disparity across a set of subgroups does not justify using those subgroups for mitigation--is well supported and has direct implications for fairness practice. The proposed KL-divergence explanation is elegant and, if it survives the target-dependence concern, would give practitioners a principled way to choose subgroupings. The counter-intuitive insight that fairness with respect to one set of subgroups can be improved by mitigating on another set is clearly demonstrated and likely to influence future work. The main weaknesses are the arbitrary choice of the uniform target distribution in the KL analysis and the lack of train-error verification, both of which affect the strength of the explanatory claim rather than the empirical findings themselves.

major comments (2)
  1. [5.3, Table 3, Appendix E] The KL-divergence explanation is computed relative to the target Punbiased = uniform over the eight (Y,S,A) atoms. This target is one of many distributions that satisfy the paper's own definition of unbiasedness (Y independent of A and S); it additionally imposes A ⊥ S, which is not entailed by that definition. In the motivating chest X-ray scenario (Olesen et al., 2024), the presence of chest drains/ECG wires (A) and sex (S) are plausibly correlated in the deployment population, so the uniform target is not self-evidently the right one. Because the minimum KL values in Table 3 are all computed relative to this chosen target, the resulting rankings (e.g., AY vs SY) are not objective properties of the subgroupings; under another valid unbiased target that preserves, say, the training-time P(A,S) marginal or a different P(A,S) marginal, the ranking can change, and the Section 5.3 explanation would no longer predict the empirical pattern. The authors acknowledge in Section 5.3 that the unbiased distribution is assumed known, but this does not address the uniform-atom target dependence. I ask the authors to (a) justify the uniform target for the medical motivating example, or (b) demonstrate that the ranking of subgroupings is invariant across a plausible family of unbiased targets (e.g., all P(Y)P(A,S) with a range of P(A,S) marginals), or (c) explicitly restrict the explanatory claim to settings where the deployment distribution is uniform over (Y,S,A). Without one of these, the paper's central explanation is not yet established beyond the specific semi-synthetic setting.
  2. [5.3] The argument that differences in the generalization upper bound are dominated by the KL term rests on the assertion that "all our models reach a similarly low train error" (Section 5.3). Nowhere in the main text or appendices is train accuracy/loss reported for the 306 trained models across the four methods and all subgroupings. If, for example, some subgroupings lead to higher training loss (e.g., CFair with S subgroups), the first term in the bound could differ substantially across models, potentially confounding the reported correlation between KL divergence and test AUC. The authors should report training-set performance (at least a summary across methods and subgroupings) or otherwise verify that the train-error term is indeed comparable. This is necessary to support the claim that the KL divergence drives the observed performance differences.
minor comments (6)
  1. [Table 1] The notation "P(Y=0,A=0|S=0)=P(Y=1,A=1|S=0)=0.95" is not a valid joint statement because the two conditional events are disjoint; it should be written as P(A=Y|S=0)=0.95, or as separate joint probabilities. This matters because Table 1 is the definition of the data-generating distribution.
  2. [5.3] The text "Pinkser's inequality" should be "Pinsker's inequality".
  3. [4.1] The phrase "Civil comments" should be "CivilComments" for consistency with the dataset name.
  4. [E.3] The text "scipy.special.rel entr" contains a stray space; also give a proper reference for scipy or omit the function name.
  5. [5.2] The explanation for why random subgroups are detrimental ("the methods themselves are sub-optimal relative to ERM on the same distribution") is vague; if random subgroups approximate the full training distribution, one would expect performance close to ERM, so the observed drop deserves more comment.
  6. [Abstract] The abstract's "Through theoretical analysis, we explain these phenomena" is broader than what Section 5.3 delivers, since the KL analysis is only applied to gDRO and resampling; consider wording like "for reweighting methods" in the abstract.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the KL-divergence explanation is computed from the known data-generating distributions, not fitted to the observed AUC values; self-citations are framing only.

full rationale

We found no circular steps. The paper's central explanatory quantity, min_w KL(P_w_train || P_unbiased), is computed analytically from the stipulated data-generating distributions (Section 3.2 and Appendix E) rather than fitted to the observed AUC values. The empirical AUCs come from 306 trained models, while the KL values come from algebra over the (Y,S,A) probability vectors; no parameter in the KL calculation is tuned to match the target results. The high correlations in Section 5.3 and Figures 5/F10 are therefore genuine empirical associations, not identities by construction. The 'unbiased test distribution' is defined as uniform over the eight atoms; this is an explicit modeling assumption about the deployment setting, and the paper acknowledges in Sections 5.3 and 5.6 that full knowledge of the test distribution and the restriction of divergence to P(Y,S,A) differences are rarely available in practice. That limits the diagnostic's applicability and external validity, but it does not make the derivation circular. Self-citations (Jones et al. 2024/2025, Alloula et al. 2024) are used for framing and background; the load-bearing argument that subgroup choice changes mitigation outcomes and that proximity to the unbiased target predicts outcomes is supported by the paper's own controlled experiments and by external theory (Ben-David et al. 2010, Zhou et al. 2021), not by a self-citation chain. The claim that subgroups showing disparities can be detrimental to mitigation is an empirical finding, not a definitional artifact.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims are demonstrated in a controlled semi-synthetic setting where the unbiased test distribution is known and uniform. The hand-chosen correlation strengths (0.95, 0.80) define the experiment, and the KL explanation assumes the divergence is fully captured by P(Y,S,A) and that the test distribution is known. These assumptions are reasonable for the controlled study but limit direct transfer to real settings.

free parameters (3)
  • P(Y=0,A=0|S=0) = P(Y=1,A=1|S=0) = 0.95
    Hand-chosen to create a strong spurious correlation in subgroup S=0, mimicking the chest X-ray scenario described in Section 3.2.
  • P(Y=0,A=0|S=1) = P(Y=1,A=1|S=1) = 0.80
    Hand-chosen for subgroup S=1, also mimicking the motivating example.
  • P_unbiased atom probabilities = 1/8 each
    The unbiased test distribution is set to uniform over (Y,S,A); all KL computations in Section 5.3 and Table 3 are relative to this chosen target.
assumptions (4)
  • domain assumption The spurious correlation between A and Y present in Ptrain is absent in the unbiased test distribution; P(Y)=P(Y|A)=P(Y|S) at test.
    Core to the problem setting (Section 3.2, Figure 1). If the test distribution still contained the correlation, the ERM baseline would not drop as claimed.
  • domain assumption All divergence between train and test is attributable to differences in P(Y,S,A), allowing representation of each distribution as a vector in R8.
    Stated in Section 5.3; required for the KL divergence calculations in Table 3.
  • standard math Pinsker's inequality and the KL-based generalization bound apply, and all models achieve similarly low train error, so divergence drives the upper bound on test error.
    Used in Section 5.3 to motivate the correlation analysis between KL divergence and unbiased test AUC.
  • domain assumption The causal graph in Figure 1, where selection on S induces the spurious correlation between A and Y, correctly describes the data generation.
    Defines the semi-synthetic setup and is cited from Jones et al. (2025).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Subgroups Matter for Robust Bias Mitigation." pith.science (2026). https://pith.science/paper/7D3L4YHP

@misc{pith2026250521363,
  author       = {Pith},
  title        = {Pith review of: Subgroups Matter for Robust Bias Mitigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7D3L4YHP}},
  note         = {Machine review of arXiv:2505.21363}
}
read the original abstract

Despite the constant development of new bias mitigation methods for machine learning, no method consistently succeeds, and a fundamental question remains unanswered: when and why do bias mitigation techniques fail? In this paper, we hypothesise that a key factor may be the often-overlooked but crucial step shared by many bias mitigation methods: the definition of subgroups. To investigate this, we conduct a comprehensive evaluation of state-of-the-art bias mitigation methods across multiple vision and language classification tasks, systematically varying subgroup definitions, including coarse, fine-grained, intersectional, and noisy subgroups. Our results reveal that subgroup choice significantly impacts performance, with certain groupings paradoxically leading to worse outcomes than no mitigation at all. Our findings suggest that observing a disparity between a set of subgroups is not a sufficient reason to use those subgroups for mitigation. Through theoretical analysis, we explain these phenomena and uncover a counter-intuitive insight that, in some cases, improving fairness with respect to a particular set of subgroups is best achieved by using a different set of subgroups for mitigation. Our work highlights the importance of careful subgroup definition in bias mitigation and presents it as an alternative lever for improving the robustness and fairness of machine learning models.

Figures

Figures reproduced from arXiv: 2505.21363 by the authors.

Figure 1
Figure 1. Causal graphs representing interactions between Y , A, and S variables in the training data and in the unbiased test data. Conditioning on selection in the training data results in spurious correlations between Y , A, and S. Coloured bars also illustrate the proportions of Y , A, and S combinations in both settings. testing distribution. Dataset bias can take many forms (as discussed in Jones et al. (2024)), but her… view at source ↗
Figure 2
Figure 2. Performance on thePunbiased in gDRO and resampling is highly dependent on the subgroups used across all four datasets. Bars represent overall change in AUC relative to the ERM baseline, with error bars indicating the standard deviation across 3 random seeds. 5.2. Groupings used for mitigation strongly impact bias mitigation performance We find that test set performance is highly dependent on the subgroups used for m… view at source ↗
Figure 4
Figure 4. Noise in (A, Y ) subgroup labels leads to a degradation in AUC for gDRO and resampling. Each dot represents mean performance on the unbiased test set for a specific grouping, with error bars indicating the standard deviation across 3 random seeds. using completely random subgroups generally worsens per￾formance relative to ERM. This is most likely because the methods themselves are sub-optimal relative to ERM on the… view at source ↗
Figures from the paper (9 more)
Figure 3
Figure 3. Figure 3: Performance on Punbiased is similar for the coarse and granular subgroups in Civil comments (mention of gender v.s. a specific male/female mention (A) and mention of any religion v.s. a specific mention of Christianity (S)). Error bars show standard deviation across 3 …
Figure 5
Figure 5. Figure 5: The ability to recover the unbiased test distribution (Punbiased) is a significant predictor of overall generalisation performance on Punbiased for gDRO and resampling. Each dot represents mean AUC on the unbiased test set for a specific grouping, with error bars indic…
Figure 6
Figure 6. Figure 6: Subgroup construction for our experiments. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Relative performance on Punbiased for different groupings in DomainInd and CFair across all four datasets. Similar trends to gDRO and resampling can be seen, where subgroupings constructed around A generally improve performance as they prevent the SC from being learnt,…
Figure 8
Figure 8. Figure 8: Effect of noise in A subgroup labels on AUC for DomainInd and CFair. Each dot represents mean performance on the unbiased test set for a specific grouping, with error bars indicating the standard deviation across 3 random seeds. 16 [PITH_FULL_IMAGE:figures/full_fig_p0…
Figure 9
Figure 9. Figure 9: Performance on the unbiased test set in gDRO and resampling and JTT is highly dependent on the subgroups used. Bars represent overall change in AUC relative to the ERM baseline, with error bars indicating the standard deviation across 3 random seeds. 17 [PITH_FULL_IMA…
Figure 10
Figure 10. Figure 10: Test AUC is highly correlated with the minimum achievable KL divergence between P w train and Punbiased across all four datasets for gDRO and resampling in CXP. Each dot represents mean performance on the unbiased test set for a specific grouping, with error bars indi…
Figure 11
Figure 11. Figure 11: Relationship between AUC and the minimum achievable distance to the unbiased test distribution (Punbiased) for gDRO and resampling in MNIST with a weaker spurious correlation. Each dot represents mean performance on the unbiased test set for a specific grouping, with …
Figure 12
Figure 12. Figure 12: Relationship between AUC and the minimum achievable KL divergence to the unbiased test distribution (Punbiased) for gDRO and resampling in MNIST with a downsampled dataset. Trends appear similar across both dataset sizes, suggesting that the results on the other three…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

63 extracted references · 39 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Mitigating dataset bias by using per-sample gradient

    Ahn, S., Kim, S., and Yun, S.-Y. Mitigating dataset bias by using per-sample gradient. In NeurIPS 2022 Workshop on Distribution Shifts: Connecting Methods and Applications, 2022. URL https://openreview.net/forum?id=ihlU5X4SEE

    work page 2022

  3. [3]

    R., and Papie \

    Alloula, A., Mustafa, R., McGowan, D. R., and Papie \. z , B. W. On biases in a uk biobank-based retinal image classification model. In Puyol-Ant \'o n, E., Zamzmi, G., Feragen, A., King, A. P., Cheplygina, V., Ganz-Benjaminsen, M., Ferrante, E., Glocker, B., Petersen, E., Baxter, J. S. H., Rekik, I., and Eagleson, R. (eds.), Ethics and Fairness in Medica...

    work page 2024

  4. [4]

    Aminian, G., Masiha, S., Toni, L., and Rodrigues, M. R. D. Learning algorithm generalization error bounds via auxiliary distributions, 2024. URL https://arxiv.org/abs/2210.00483

    work page Pith review arXiv 2024

  5. [5]

    and Kamnitsas, K

    Anthony, H. and Kamnitsas, K. On the use of mahalanobis distance for out-of-distribution detection with neural networks for medical imaging. In Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, pp.\ 136--146. Springer Nature Switzerland, 2023. doi:10.1007/978-3-031-44336-7_14. URL https://doi.org/10.1007

    work page doi:10.1007/978-3-031-44336-7_14 2023

  6. [6]

    Equalized odds postprocessing under imperfect group information

    Awasthi, P., Kleindessner, M., and Morgenstern, J. Equalized odds postprocessing under imperfect group information. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , pp.\ 1770--1780. PMLR, June 2020

    work page 2020

  7. [7]

    BiasPruner : Debiased Continual Learning for Medical Image Classification

    Bayasi, N., Fayyad, J., Bissoto, A., Hamarneh, G., and Garbi, R. BiasPruner : Debiased Continual Learning for Medical Image Classification . In MICCAI, 2024

    work page 2024

  8. [8]

    A theory of learning from different domains

    Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. A theory of learning from different domains. Machine Learning, 79: 0 151--175, 2010. URL http://www.springerlink.com/content/q6qk230685577n52/

    work page 2010

Show all 63 references
  1. [9]

    Nuanced metrics for measuring unintended bias with real data for text classification

    Borkan, D., Dixon, L., Sorensen, J., Thain, N., and Vasserman, L. Nuanced metrics for measuring unintended bias with real data for text classification. CoRR, abs/1903.04561, 2019. URL http://arxiv.org/abs/1903.04561

    1903 arXiv

  2. [10]

    and Gebru, T

    Buolamwini, J. and Gebru, T. Gender Shades : Intersectional Accuracy Disparities in Commercial Gender Classification . In Proceedings of the 1st Conference on Fairness , Accountability and Transparency , pp.\ 77--91. PMLR, January 2018

    2018

  3. [11]

    M., Sarro, F., and Harman, M

    Chen, Z., Zhang, J. M., Sarro, F., and Harman, M. A comprehensive empirical study of bias mitigation methods for machine learning classifiers. ACM Trans. Softw. Eng. Methodol., 32 0 (4), May 2023. ISSN 1049-331X. doi:10.1145/3583561. URL https://doi.org/10.1145/3583561

    2023 doi

  4. [12]

    Estimating generalization under distribution shifts via domain-invariant representations

    Chuang, C.-Y., Torralba, A., and Jegelka, S. Estimating generalization under distribution shifts via domain-invariant representations. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org, 2020

    2020

  5. [13]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 248--255, 2009. doi:10.1109/CVPR.2009.5206848

    2009

  6. [14]

    BERT: pre-training of deep bidirectional transformers for language understanding

    Devlin, J., Chang, M., Lee, K., and Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805, 2018. URL http://arxiv.org/abs/1810.04805

    2018 arXiv

  7. [15]

    How does distribution matching help domain generalization: An information-theoretic analysis, 2024

    Dong, Y., Gong, T., Chen, H., Song, S., Zhang, W., and Li, C. How does distribution matching help domain generalization: An information-theoretic analysis, 2024. URL https://arxiv.org/abs/2406.09745

    2024 arXiv

  8. [16]

    Fairness through awareness

    Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference , ITCS '12, pp.\ 214--226, New York, NY, USA, January 2012. Association for Computing Machinery. ISBN 978...

    2012

  9. [17]

    and Lempitsky, V

    Ganin, Y. and Lempitsky, V. Unsupervised domain adaptation by backpropagation. In Bach, F. and Blei, D. (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp.\ 1180--1189, Lille, France, 07--09 J...

    2015

  10. [18]

    Ghosal, S. S. and Li, Y. Distributionally Robust Optimization with Probabilistic Group . Proceedings of the AAAI Conference on Artificial Intelligence, 37 0 (10): 0 11809--11817, June 2023. ISSN 2374-3468. doi:10.1609/aaai.v37i10.26394

    2023 doi

  11. [19]

    and Zou, D

    Han, Y. and Zou, D. Improving group robustness on spurious correlation requires preciser group inference. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=KycvgOCBBR

    2024

  12. [20]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016

    2016

  13. [21]

    Multicalibration: Calibration for the ( C omputationally-identifiable) masses

    Hebert-Johnson, U., Kim, M., Reingold, O., and Rothblum, G. Multicalibration: Calibration for the ( C omputationally-identifiable) masses. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine...

    1939

  14. [22]

    Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K. Q. Densely connected convolutional networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 2261--2269, 2017. doi:10.1109/CVPR.2017.243

    2017 doi

  15. [23]

    Y., Arjovsky, M., Pezeshki, M., and Lopez-Paz , D

    Idrissi, B. Y., Arjovsky, M., Pezeshki, M., and Lopez-Paz , D. Simple data balancing achieves competitive worst-group-accuracy. In Proceedings of the First Conference on Causal Learning and Reasoning , pp.\ 336--351. PMLR, June 2022

    2022

  16. [24]

    A., Halabi, S

    Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., Seekins, J., Mong, D. A., Halabi, S. S., Sandberg, J. K., Jones, R., Larson, D. B., Langlotz, C. P., Patel, B. N., Lungren, M. P., and Ng, A. Y. Chexpert...

    2019

  17. [25]

    Izmailov, P., Kirichenko, P., Gruver, N., and Wilson, A. G. On feature learning in the presence of spurious correlations. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISB...

    2022

  18. [26]

    Improving subgroup robustness via data selection

    Jain, S., Hamidieh, K., Georgiev, K., Ilyas, A., Ghassemi, M., and Madry, A. Improving subgroup robustness via data selection. In Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., and Zhang, C. (eds.), Advances in Neural Information Processing Systems,...

    2024

  19. [27]

    C., De Sousa Ribeiro, F., Oktay, O., McCradden, M., and Glocker, B

    Jones, C., Castro, D. C., De Sousa Ribeiro, F., Oktay, O., McCradden, M., and Glocker, B. A causal perspective on dataset bias in machine learning for medical imaging. Nature Machine Intelligence, 6 0 (2): 0 138--146, February 2024. ISSN 2522-5839. doi:10.1038/s42256-024-00797-8

    2024 doi

  20. [28]

    Jones, C., Ribeiro, F. d. S., Roschewitz, M., Castro, D. C., and Glocker, B. Rethinking fair representation learning for performance-sensitive tasks. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=pBZntPrdrI

    2025

  21. [29]

    Kearns, M., Neel, S., Roth, A., and Wu, Z. S. Preventing Fairness Gerrymandering : Auditing and Learning for Subgroup Fairness . In Proceedings of the 35th International Conference on Machine Learning , pp.\ 2564--2572. PMLR, July 2018

    2018

  22. [30]

    Improving Robustness to Multiple Spurious Correlations by Multi-Objective Optimization

    Kim, N., Kang, J., Ahn, S., Ok, J., and Kwak, S. Improving Robustness to Multiple Spurious Correlations by Multi-Objective Optimization . In Proceedings of the 41st International Conference on Machine Learning , pp.\ 24040--24058. PMLR, July 2024

    2024

  23. [31]

    Kirichenko, P., Izmailov, P., and Wilson, A. G. Last layer re-training is sufficient for robustness to spurious correlations. In ICLR, 2023. URL https://arxiv.org/abs/2204.02937

    2023 arXiv

  24. [32]

    Udis: Unsupervised discovery of bias in deep visual recognition models

    Krishnakumar, A., Prabhu, V., Sudhakar, S., and Hoffman, J. Udis: Unsupervised discovery of bias in deep visual recognition models. In BMVC, pp.\ 143, 2021. URL https://www.bmvc2021-virtualconference.com/assets/papers/0362.pdf

    2021

  25. [33]

    Gradient-based learning applied to document recognition

    Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, November 1998. ISSN 1558-2256. doi:10.1109/5.726791

    1998 doi

  26. [34]

    C., Xu, C., and Ibrahim, M

    Li, Z., Evtimov, I., Gordo, A., Hazirbas, C., Hassner, T., Ferrer, C. C., Xu, C., and Ibrahim, M. A Whac-A-Mole Dilemma : Shortcuts Come in Multiples Where Mitigating One Amplifies Others . 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 20071-...

    2023

  27. [35]

    Z., Haghgoo, B., Chen, A

    Liu, E. Z., Haghgoo, B., Chen, A. S., Raghunathan, A., Koh, P. W., Sagawa, S., Liang, P., and Finn, C. Just train twice: Improving group robustness without training group information. In Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference on Machin...

    2021

  28. [36]

    Deep learning face attributes in the wild

    Liu, Z., Luo, P., Wang, X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015 a

    2015

  29. [37]

    Deep learning face attributes in the wild

    Liu, Z., Luo, P., Wang, X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), ICCV '15, pp.\ 3730–3738, USA, 2015 b . IEEE Computer Society. ISBN 9781467383912. doi:10.1109/ICCV.2015.425....

    2015 doi

  30. [38]

    Domain adaptation: Learning bounds and algorithms

    Mansour, Y., Mohri, M., and Rostamizadeh, A. Domain adaptation: Learning bounds and algorithms. CoRR, abs/0902.3430, 2009. URL http://arxiv.org/abs/0902.3430

    2009 arXiv

  31. [39]

    ViG-Bias : Visually Grounded Bias Discovery and Mitigation , pp.\ 414–429

    Marani, B.-E., Hanini, M., Malayarukil, N., Christodoulidis, S., Vakalopoulou, M., and Ferrante, E. ViG-Bias : Visually Grounded Bias Discovery and Mitigation , pp.\ 414–429. Springer Nature Switzerland, November 2024. ISBN 9783031732027. doi:10.1007/978-3-031-73202-7_24. URL ...

    2024 doi

  32. [40]

    S., Gohari, A., Yassaee, M

    Masiha, M. S., Gohari, A., Yassaee, M. H., and Aref, M. R. Learning under distribution mismatch and model misspecification. CoRR, abs/2102.05695, 2021. URL https://arxiv.org/abs/2102.05695

    2021 arXiv

  33. [41]

    A survey on bias and fairness in machine learning

    Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., and Galstyan, A. A survey on bias and fairness in machine learning. ACM Comput. Surv., 54 0 (6), July 2021. ISSN 0360-0300. doi:10.1145/3457607. URL https://doi.org/10.1145/3457607

    2021 doi

  34. [42]

    Coarse race data conceals disparities in clinical risk score performance

    Movva, R., Shanmugam, D., Hou, K., Pathak, P., Guttag, J., Garg, N., and Pierson, E. Coarse race data conceals disparities in clinical risk score performance. In Machine Learning for Healthcare Conference, pp.\ 443--472. PMLR, 2023

    2023

  35. [43]

    T., Tran, T., Gal, Y., Torr, P

    Nguyen, A. T., Tran, T., Gal, Y., Torr, P. H. S., and Baydin, A. G. Kl guided domain adaptation, 2022. URL https://arxiv.org/abs/2106.07780

    2022 arXiv

  36. [44]

    Slicing Through Bias : Explaining Performance Gaps in Medical Image Analysis Using Slice Discovery Methods , pp.\ 3–13

    Olesen, V., Weng, N., Feragen, A., and Petersen, E. Slicing Through Bias : Explaining Performance Gaps in Medical Image Analysis Using Slice Discovery Methods , pp.\ 3–13. Springer Nature Switzerland, October 2024. ISBN 9783031727870. doi:10.1007/978-3-031-72787-0_1. URL http:...

    2024 doi

  37. [45]

    Y., Jung, C., Lee, S., Ye, J

    Park, G. Y., Jung, C., Lee, S., Ye, J. C., and Lee, S. W. Self-supervised debiasing using low rank regularization. In 2024 IEEE / CVF Conference on Computer Vision and Pattern Recognition ( CVPR ) , 2024. doi:10.48550/ARXIV.2210.05248

    2024 doi arXiv

  38. [46]

    H., and Feragen, A

    Petersen, E., aPETERSEN2023100790nd Melanie Ganz, S. H., and Feragen, A. The path toward equal performance in medical machine learning. Patterns, 4 0 (7): 0 100790, 2023. ISSN 2666-3899. doi:https://doi.org/10.1016/j.patter.2023.100790. URL https://www.sciencedirect.com/scienc...

    2023

  39. [47]

    Gradient Starvation : A Learning Proclivity in Neural Networks , November 2021

    Pezeshki, M., Kaba, S.-O., Bengio, Y., Courville, A., Precup, D., and Lajoie, G. Gradient Starvation : A Learning Proclivity in Neural Networks , November 2021

    2021

  40. [48]

    Discovering environments with XRM

    Pezeshki, M., Bouchacourt, D., Ibrahim, M., Ballas, N., Vincent, P., and Lopez-Paz, D. Discovering environments with XRM . In ICML, 2024. URL https://openreview.net/forum?id=IhWtRwIbos

    2024

  41. [49]

    A., Echeveste, R., and Ferrante, E

    Ricci Lara, M. A., Echeveste, R., and Ferrante, E. Addressing fairness in artificial intelligence for medical imaging. Nature Communications, 13 0 (1): 0 4581, August 2022. ISSN 2041-1723. doi:10.1038/s41467-022-32186-3

    2022 doi

  42. [50]

    W., Hashimoto, T

    Sagawa*, S., Koh*, P. W., Hashimoto, T. B., and Liang, P. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=ryxGuJrFvS

    2020

  43. [51]

    N., and Chiappa, S

    Schrouff, J., Bellot, A., Rannen-Triki, A., Malek, A., Albuquerque, I., Gretton, A., D'Amour, A. N., and Chiappa, S. Mind the graph when balancing data for fairness or robustness. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https:...

    2024

  44. [52]

    An Investigation of Critical Issues in Bias Mitigation Techniques

    Shrestha, R., Kafle, K., and Kanan, C. An Investigation of Critical Issues in Bias Mitigation Techniques . In 2022 IEEE / CVF Winter Conference on Applications of Computer Vision ( WACV ) , pp.\ 2512--2523, Waikoloa, HI, USA, January 2022. IEEE. ISBN 978-1-66540-915-5. doi:10....

    2022

  45. [53]

    For robust worst-group accuracy, ignore group annotations

    Stromberg, N., Ayyagari, R., Welfert, M., Koyejo, S., Nock, R., and Sankar, L. For robust worst-group accuracy, ignore group annotations. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=l8E68fD6yp

    2024

  46. [54]

    Robust Optimization for Fairness with Noisy Protected Groups

    Wang, S., Guo, W., Narasimhan, H., Cotter, A., Gupta, M., and Jordan, M. Robust Optimization for Fairness with Noisy Protected Groups . In Advances in Neural Information Processing Systems , volume 33, pp.\ 5190--5203. Curran Associates, Inc., 2020 a

    2020

  47. [55]

    Wang, X., Saxon, M., Li, J., Zhang, H., Zhang, K., and Wang, W. Y. Causal balancing for domain generalization. In ICLR, 2023. URL https://arxiv.org/abs/2206.05263

    2023 arXiv

  48. [56]

    C., Genova, K., Nair, P., Hata, K., and Russakovsky, O

    Wang, Z., Qinami, K., Karakozis, I. C., Genova, K., Nair, P., Hata, K., and Russakovsky, O. Towards Fairness in Visual Recognition : Effective Strategies for Bias Mitigation . In 2020 IEEE / CVF Conference on Computer Vision and Pattern Recognition ( CVPR ) , pp.\ 8916--8925, ...

    2020

  49. [57]

    Weng, N., Bigdeli, S., Petersen, E., and Feragen, A. Are sex-based physiological differences the cause of gender bias for chest x-ray diagnosis? In Clinical Image-Based Procedures, Fairness of AI in Medical Imaging, and Ethical and Philosophical Issues in Medical Imaging: 12th...

    2023

  50. [58]

    H., Aickelin, U., and Zhu, J

    Wu, X., Manton, J. H., Aickelin, U., and Zhu, J. On the generalization for transfer learning: An information-theoretic analysis, 2024. URL https://arxiv.org/abs/2207.05377

    2024 arXiv

  51. [59]

    Intersectional Unfairness Discovery

    Xu, G., Chen, Q., Ling, C., Wang, B., and Shui, C. Intersectional Unfairness Discovery . In Proceedings of the 41st International Conference on Machine Learning , pp.\ 54888--54917. PMLR, July 2024

    2024

  52. [60]

    Zhao, H., Coston, A., Adel, T., and Gordon, G. J. Conditional learning of fair representations. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=Hkekl0NFPr

    2020

  53. [61]

    Examining and Combating Spurious Features under Distribution Shift

    Zhou, C., Ma, X., Michel, P., and Neubig, G. Examining and Combating Spurious Features under Distribution Shift . In Proceedings of the 38th International Conference on Machine Learning , pp.\ 12857--12867. PMLR, July 2021

    2021

  54. [62]

    Leveling down in computer vision: Pareto inefficiencies in fair deep classifiers

    Zietlow, D., Lohaus, M., Balakrishnan, G., Kleindessner, M., Locatello, F., Scholkopf, B., and Russell, C. Leveling down in computer vision: Pareto inefficiencies in fair deep classifiers. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 10400--...

    2022

  55. [63]

    Medfair: Benchmarking fairness for medical imaging

    Zong, Y., Yang, Y., and Hospedales, T. Medfair: Benchmarking fairness for medical imaging. In International Conference on Learning Representations (ICLR), 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.