REVIEW 3 major objections 6 minor 32 references
AI fairness research is concentrated in a few countries, most of all in the foundational domain that supplies the rest of the field's definitions and benchmarks.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-11 05:35 UTC pith:BAZHUXIU
load-bearing objection Solid first field-wide map of AI-bias research; concentration is real inside the sampled frame, strongest in the foundational domain, and the sampling caveat is already owned by the authors. the 3 major comments →
Whose fairness? Structural concentration in AI bias research
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
AI bias research is structurally concentrated, and the concentration is greatest geographically in general fairness and bias mitigation—the largest, most-cited, semantically cross-cutting domain that supplies the definitions, benchmarks, and mitigation frameworks used across health AI, LLMs and NLP, recommender systems, and graph-based fairness. United States authors lead publication output and collaboration networks in every domain and account for more than half of first-authored work in that foundational domain; low- and middle-income countries remain largely absent; and citation influence is highly skewed (median 9, mean 93.5).
What carries the argument
A 692-paper corpus across five manually assigned thematic domains, analyzed with bibliometric measures (all-author and first-author country and institution counts, collaboration networks, Gini coefficients, citation distributions) and semantic clustering of abstracts (Sentence-BERT embeddings, UMAP, HDBSCAN), which identifies general fairness as the cross-cutting foundational domain where concentration is strongest.
Load-bearing premise
The measured concentration is taken to reflect the true production structure of the field, even though the corpus comes mainly from English-language, Global-North-indexed databases and may under-represent work from other languages and venues.
What would settle it
A parallel corpus built with equal coverage of major non-English and Global-South venues that reverses or erases United States dominance and the relative concentration in the general-fairness domain would falsify the central claim about structural concentration in the field as a whole.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents a bibliometric and semantic analysis of 692 AI-bias publications (2015–2026) across five manually assigned thematic domains, arguing that research production is structurally concentrated by country, institution, and author, and that this concentration is strongest in General Fairness & Bias Mitigation—the largest, most-cited, and semantically cross-cutting domain that supplies definitions and benchmarks to application areas. Using all-author and first-author counts, co-authorship networks, Gini coefficients, non-parametric citation tests, and Sentence-BERT + UMAP + HDBSCAN clustering (with a default-UMAP robustness check), the authors report US leadership (e.g., 58.3% of first-authored General Fairness papers), sparse LMIC participation and collaboration, and highly skewed citations (median 9, mean 93.5). They release an interactive atlas and public data/code for ongoing monitoring.
Significance. If the measured concentration is taken as a fair description of the indexed literature that sets fairness standards, the paper makes a useful, field-level contribution: it quantifies who produces the methodological layer that application domains inherit, rather than only documenting underrepresentation within a single application area. Strengths include dual all-author/first-author reporting, appropriate non-parametric statistics with bootstrap CIs, unsupervised clustering validated post-hoc (ARI/NMI) without using domain labels in the pipeline, a default-UMAP robustness check, and public corpus, code, and interactive atlas. The work is timely for cs.CY and fairness research policy, even if the strongest epistemic claim about non-generalizability remains partly inferential.
major comments (3)
- [Methods (Corpus construction); Limitations; Abstract; Discussion] The central claim is framed as field-wide structural concentration (Abstract; Introduction; Discussion), but the corpus is built from English-language, Global-North-indexed sources (IEEE Xplore, ACM DL, Scopus, ScienceDirect, Engineering Village, FAccT, snowballing; Methods, Corpus construction). Limitations correctly notes under-representation of non-English and regional venues, yet the paper asserts that broader coverage is “unlikely to reverse” the pattern without a sensitivity check. Because the load-bearing inference is that concentration is greatest precisely in the foundational domain the rest of the field inherits, the manuscript should either (i) re-scope claims explicitly to the well-indexed English literature that currently sets mainstream benchmarks, or (ii) add a concrete sensitivity analysis (e.g., regional venues, non-English indexes, or stratified snowballing from Global-
- [Abstract; Discussion (esp. paragraphs on epistemic influence and implications)] The Discussion links geographic concentration in General Fairness to a risk that mitigation methods “may not generalize,” while also stating that the data “do not directly establish” reduced generalizability and citing the authors’ Nepali LLM study [9] as illustration. That hedge is appropriate but under-weighted relative to the Abstract and closing paragraphs, which present non-generalizability as a field-wide concern flowing from the bibliometric pattern. Tighten the causal language: treat concentration as a measured structural fact within the sampled frame, and treat non-generalizability as a motivated hypothesis requiring external validation designs (multi-site benchmarks, reporting of validation populations), not as a demonstrated consequence of the Gini/country shares alone.
- [Methods (Corpus construction); Results (Domain distribution); Fig. 3] Domain assignment is manual and load-bearing for the claim that concentration is greatest in the foundational domain (Results, Domain distribution; Fig. 1; Fig. 3). Semantic clustering supports coherence (Fig. 6; ARI/NMI; robustness check), but Methods does not report inter-rater reliability, double-coding fraction, or a decision protocol for multi-domain papers. Given that General Fairness is defined as cross-cutting and is the domain where US dominance is strongest (58.3% first-author), please report how borderline papers were assigned and whether reassignment of a plausible fraction of multi-topic papers would change the domain-wise country shares or the “most concentrated foundational domain” ranking.
minor comments (6)
- [Abstract] Abstract: “the AI bias research are structurally concentrated” — subject–verb agreement; also “mean =93.5” needs a space.
- [Fig. 2c] Fig. 2c collaboration heatmap is dense; a thresholded network or top-k edges would improve readability without changing the Western-hub message.
- [Fig. 4 caption] Institutional heatmaps (Fig. 4) report percentages among the top-20 institutions shown; state this more prominently in the caption so readers do not read them as shares of the full institutional population.
- [Results (Country-level); Methods (Statistics)] Chi-square uses 11 countries × 5 domains with Monte Carlo p-values (Results); report the exact country list and the ≥10 first-authored-paper threshold in one place for reproducibility.
- [Methods (Author and institution counting)] ORCID-based author deduplication was not applied (Methods); briefly discuss possible inflation of unique-author counts for common names, especially for China/India comparisons.
- [Results (Temporal trends)] arXiv version date in header (“6 Jul 2026”) and incomplete 2026 counts are fine if noted; ensure temporal claims in the text always flag incomplete-year status (already partly done in Results).
Circularity Check
No circularity: empirical bibliometric measurement of an external 692-paper corpus; manual domains validated post-hoc by unsupervised clustering that never used domain labels as input.
full rationale
The paper’s load-bearing chain is observational, not definitional or fitted-then-predicted. Corpus construction (keyword search + snowballing + FAccT + manual screening) yields 692 papers; country/institution/first-author counts, Gini coefficients, collaboration edges, Kruskal–Wallis/Dunn citation tests, and chi-square domain-by-country tests are computed directly from that external metadata. Semantic clustering encodes abstracts with Sentence-BERT, projects with UMAP, and clusters with HDBSCAN; domain labels are applied only after clustering to compute ARI/NMI, so the four clusters are not forced by the five manual categories. The sole self-citation ([9], Nepali LLM bias) appears only as an illustrative risk example in Discussion/Introduction and is not an input to any count, Gini, network, or cluster. No parameter is fitted to a subset and then re-presented as a prediction; no uniqueness theorem or ansatz is imported from the authors’ prior work to force the result. Sampling-frame limitations (English/Global-North databases) affect external validity of the concentration claim but do not make the reported statistics circular by construction. Honest non-finding: score 0.
Axiom & Free-Parameter Ledger
free parameters (2)
- UMAP n_neighbors / min_dist and HDBSCAN min_cluster_size / min_samples =
n_neighbors=30, min_dist=0.0, min_cluster_size=20, min_samples=10
- Country inclusion threshold for chi-square and Spearman =
≥10 papers
axioms (5)
- domain assumption Papers retrieved from English-language Global-North databases plus FAccT and snowballing constitute a usable sample of AI-bias research for measuring structural concentration.
- domain assumption Manual assignment of each paper to one of five primary thematic domains is a valid analytic partition.
- domain assumption First-author country/institution is a reasonable proxy for research leadership; all-author counts proxy collaborative participation.
- standard math Standard bibliometric indicators (publication counts, co-authorship edges, OpenAlex citation counts, Gini) capture the field's documented structure.
- domain assumption Concentration of production in the general-fairness domain raises a generalizability risk for methods adopted elsewhere.
invented entities (2)
-
Five thematic domains (General Fairness & Bias Mitigation, Health & Clinical AI, LLMs & NLP, Recommender Systems, Graph-Based Fairness)
no independent evidence
-
Interactive Bias Atlas (biasatlas.cair-nepal.org)
independent evidence
read the original abstract
Artificial intelligence increasingly mediates consequential decisions in healthcare, law, and public services, and the field has responded with an extensive methodology for measuring and mitigating bias. Yet the fairness definitions, benchmarks, and debiasing frameworks on which this methodology rests are treated as universal while being produced by a research community whose composition has never been characterized. We show that the AI bias research are structurally concentrated, and that this concentration is greatest, geographically, in precisely the domain the rest of the field inherits from. Analyzing 692 publications spanning five thematic domains, combining bibliometric analysis with semantic clustering, we find that research activity is dominated by a small set of countries, institutions, and authors, with the United States leading publication output and collaboration networks across every domain and most strongly in general fairness and bias mitigation, the largest, most-cited domain with meaningful representation across all four semantic clusters. Low- and middle-income countries remain largely absent from the community and its collaboration networks, and citation influence is highly skewed (median = 9; mean =93.5 ), indicating that a small fraction of publications disproportionately shapes the field. Because the general-fairness domain supplies the definitions and benchmarks that application areas apply, concentration of research effort in this foundational domain propagates across AI bias research as a whole - raising the concern that mitigation methods developed and validated within a narrow set of contexts may not generalize to all populations and settings where AI is deployed. We provide an interactive atlas for continuous monitoring of the field's structure.
Reference graph
Works this paper leans on
-
[1]
Microchemical Journal205, 111307 (2024) https://doi.org/10.1016/j.microc.2024.111307
Ahmed Taha, B., Kadhim, A.C., Addie, A.J., Haider, A.J., Azzahrani, A.S., Raizada, P., Rustagi, S., Chaudhary, V., Arsad, N.: Advancing cancer diagnostics through multifaceted optical biosensors supported by nanomaterials and artifi- cial intelligence: A panoramic outlook. Microchemical Journal205, 111307 (2024) https://doi.org/10.1016/j.microc.2024.111307
-
[2]
Drug Discovery Today29(6), 103992 (2024) https://doi.org/10.1016/j
Gangwal, A., Lavecchia, A.: Unleashing the power of generative ai in drug dis- covery. Drug Discovery Today29(6), 103992 (2024) https://doi.org/10.1016/j. drudis.2024.103992
doi:10.1016/j 2024
-
[3]
Science Advances4(1), 5580 (2018) https://doi.org/10.1126/sciadv.aao5580 16
Dressel, J., Farid, H.: The accuracy, fairness, and limits of predicting recidivism. Science Advances4(1), 5580 (2018) https://doi.org/10.1126/sciadv.aao5580 16
-
[4]
Sci6(1), 3 (2023) https://doi.org/10.3390/ sci6010003
Ferrara, E.: Fairness and bias in artificial intelligence: A brief survey of sources, impacts, and mitigation strategies. Sci6(1), 3 (2023) https://doi.org/10.3390/ sci6010003
2023
-
[5]
WIREs Data Mining and Knowledge Discovery10(3), 1356 (2020) https://doi.org/10.1002/widm.1356
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M.- E., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., Kompatsiaris, I., Kinder-Kurlanda, K., Wagner, C., Karimi, F., Fernandez, M., Alani, H., Berendt, B., Kruegel, T., Heinze, C., Broelemann, K., Kasneci, G., Tiropanis, T., Staab, S.: Bias in data-driven artificial in...
-
[6]
Giloni, A., Grolman, E., Hagemann, T., Fromm, R., Fischer, S., Elovici, Y., Shabtai, A.: Benn: Bias estimation using a deep neural network. IEEE Trans- actions on Neural Networks and Learning Systems35(1), 117–131 (2024) https: //doi.org/10.1109/TNNLS.2022.3172365
-
[7]
The Lancet Digital Health6(1), 12–22 (2024) https://doi.org/10.1016/S2589-7500(23)00225-X
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J.A., Celi, L.A., Gichoya, J., Juraf- sky, D., Szolovits, P., Bates, D.W., Abdulnour, R.-E.E., Butte, A.J., Alsentzer, E.: Assessing the potential of gpt-4 to perpetuate racial and gender biases in health care: a model evaluation study. The Lancet Digital Health6(1), 12–22 (2024) https://doi.org/10.1016/S2589-7...
-
[8]
Association for Computational Linguistics, Abu Dhabi, UAE (2025)
Lin, L., Wang, L., Guo, J., Wong, K.-F.: Investigating Bias in LLM-Based Bias Detection: Disparities betwceen LLMs and Human Perception. Association for Computational Linguistics, Abu Dhabi, UAE (2025). https://aclanthology.org/ 2025.coling-main.709/
2025
-
[9]
https://arxiv.org/abs/2603.07792
Pandey, A., Chhetri, T.R.: Dual-Metric Evaluation of Social Bias in Large Lan- guage Models: Evidence from an Underrepresented Nepali Cultural Context (2026). https://arxiv.org/abs/2603.07792
-
[10]
Artificial Intelligence and Law (2024) https://doi.org/10.1007/s10506-024-09389-8
Engel, C., Linhardt, L., Schubert, M.: Code is law: how compas affects the way the judiciary handles the risk of recidivism. Artificial Intelligence and Law (2024) https://doi.org/10.1007/s10506-024-09389-8
-
[11]
In: Proceedings of the Conference on Fairness, Accountability, and Transparency
Hutchinson, B., Mitchell, M.: 50 years of test (un)fairness: Lessons for machine learning. In: Proceedings of the Conference on Fairness, Accountability, and Transparency. FAT* ’19, pp. 49–58. Association for Computing Machin- ery, New York, NY, USA (2019). https://doi.org/10.1145/3287560.3287600 . https://doi.org/10.1145/3287560.3287600
-
[12]
ACM Computing Surveys54(6), 1–35 (2021) https://doi.org/10.1145/3457607
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM Computing Surveys54(6), 1–35 (2021) https://doi.org/10.1145/3457607
doi:10.1145/3457607 2021
-
[13]
ACM Computing 17 Surveys56(7), 1–38 (2024) https://doi.org/10.1145/3616865
Caton, S., Haas, C.: Fairness in machine learning: A survey. ACM Computing 17 Surveys56(7), 1–38 (2024) https://doi.org/10.1145/3616865
doi:10.1145/3616865 2024
-
[14]
ACM Journal on Responsible Computing1(2), 1–52 (2024) https://doi.org/10.1145/3631326
Hort, M., Chen, Z., Zhang, J.M., Harman, M., Sarro, F.: Bias mitigation for machine learning classifiers: A comprehensive survey. ACM Journal on Responsible Computing1(2), 1–52 (2024) https://doi.org/10.1145/3631326
-
[15]
Pessach, D., Shmueli, E.: A review on fairness in machine learning. ACM Comput. Surv.55(3) (2022) https://doi.org/10.1145/3494672
doi:10.1145/3494672 2022
-
[16]
ACM Computing Surveys55(13s), 1–37 (2023) https://doi.org/10.1145/3597199
Tang, Z., Zhang, J., Zhang, K.: What-is and how-to for fairness in machine learn- ing: A survey, reflection, and perspective. ACM Computing Surveys55(13s), 1–37 (2023) https://doi.org/10.1145/3597199
-
[17]
Li, Y., Du, M., Song, R., Wang, X., Wang, Y.: A Survey on Fairness in Large Language Models. arXiv (2023). https://doi.org/10.48550/ARXIV.2308.10149
-
[18]
Computational Linguistics50(3), 1097–1179 (2024) https://doi.org/10
Gallegos, I.O., Rossi, R.A., Barrow, J., Tanjim, M.M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., Ahmed, N.K.: Bias and fairness in large language models: A survey. Computational Linguistics50(3), 1097–1179 (2024) https://doi.org/10. 1162/coli a 00524
2024
-
[19]
ACM SIGKDD Explorations Newsletter26(1), 34–48 (2024) https://doi
Chu, Z., Wang, Z., Zhang, W.: Fairness in large language models: A taxonomic survey. ACM SIGKDD Explorations Newsletter26(1), 34–48 (2024) https://doi. org/10.1145/3682112.3682117
-
[20]
In: Proceedings of the 30th International Conference on Intelligent User Interfaces, pp
Dudy, S., Tholeti, T., Ramachandranpillai, R., Ali, M., Li, T.J.-J., Baeza-Yates, R.: Unequal Opportunities: Examining the Bias in Geographical Recommen- dations by Large Language Models. In: Proceedings of the 30th International Conference on Intelligent User Interfaces, pp. 1499–1516. ACM, Cagliari Italy (2025). https://doi.org/10.1145/3708359.3712111
-
[21]
Journal of Nonverbal Behavior48(2), 323– 344 (2024) https://doi.org/10.1007/s10919-024-00454-z
Pang, H.T., Zhou, X., Chu, M.: Cross-cultural differences in using nonverbal behaviors to identify indirect replies. Journal of Nonverbal Behavior48(2), 323– 344 (2024) https://doi.org/10.1007/s10919-024-00454-z
-
[22]
PLOS ONE10(2), 1–15 (2015) https://doi.org/10.1371/journal
Uono, S., Hietanen, J.K.: Eye contact perception in the west and east: A cross- cultural study. PLOS ONE10(2), 1–15 (2015) https://doi.org/10.1371/journal. pone.0118094
doi:10.1371/journal 2015
-
[23]
https://arxiv.org/abs/2102.01265
Chan, A., Okolo, C.T., Terner, Z., Wang, A.: The Limits of Global Inclusion in AI Development (2021). https://arxiv.org/abs/2102.01265
Pith/arXiv arXiv 2021
-
[24]
Perc, M.: The matthew effect in empirical data. Journal of The Royal Society Interface11(98), 20140378 (2014) https://doi.org/ 10.1098/rsif.2014.0378 https://royalsocietypublishing.org/rsif/article- pdf/doi/10.1098/rsif.2014.0378/192325/rsif.2014.0378.pdf 18
-
[25]
Merton, R.K.: The matthew effect in science. Science 159(3810), 56–63 (1968) https://doi.org/10.1126/science.159.3810.56 https://www.science.org/doi/pdf/10.1126/science.159.3810.56
-
[26]
Nature Communications 15(1), 7527 (2024) https://doi.org/10.1038/s41467-024-51714-x
Schmallenbach, L., B¨ arnighausen, T.W., Lerchenmueller, M.J.: The global geog- raphy of artificial intelligence in life science research. Nature Communications 15(1), 7527 (2024) https://doi.org/10.1038/s41467-024-51714-x
-
[27]
PLOS Global Public Health4(1), 0002513 (2024) https://doi.org/10.1371/journal.pgph.0002513
Alberto, I.R.I., Alberto, N.R.I., Altinel, Y., Blacker, S., Binotti, W.W., Celi, L.A., Chua, T., Fiske, A., Griffin, M., Karaca, G., Mokolo, N., Naawu, D.K.N., Patscheider, J., Petushkov, A., Quion, J.M., Senteio, C., Taisbak, S., Tırnova, ˙I., Tokashiki, H., Velasquez, A., Yaghy, A., Yap, K.: A scientometric analysis of fairness in health ai literature. ...
-
[28]
Science366(6464), 447–453 (2019) https://doi.org/10.1126/science.aax2342
Obermeyer, Z., Powers, B., Vogeli, C., Mullainathan, S.: Dissecting racial bias in an algorithm used to manage the health of populations. Science366(6464), 447–453 (2019) https://doi.org/10.1126/science.aax2342
-
[29]
Bender, E.M., Gebru, T., McMillan-Major, A., Shmitchell, S.: On the dangers of stochastic parrots: Can language models be too big? In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. FAccT ’21, pp. 610–623. ACM, . (2021). https://doi.org/10.1145/3442188.3445922
-
[30]
In: Proceedings of the 3rd Innovations in Theoretical Computer Sci- ence Conference
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Sci- ence Conference. ITCS ’12, pp. 214–226. ACM, New York, NY, USA (2012). https://doi.org/10.1145/2090236.2090255
-
[31]
Cambridge University Press, Cambridge (2023)
Oliveira, G., Baggs, E.: Psychology’s WEIRD Problems. Cambridge University Press, Cambridge (2023). https://doi.org/10.1017/9781009303538
-
[32]
1: Domain distribution and temporal dynamics.(a)Distribution of 692 papers across five thematic domains.(b)Annual publication volume by domain (2015–2026)
Henrich, J., Heine, S.J., Norenzayan, A.: The weirdest people in the world? Behavioral and Brain Sciences33(2-3), 61–83 (2010) https://doi.org/10.1017/ s0140525x0999152x 19 (a) (b) Fig. 1: Domain distribution and temporal dynamics.(a)Distribution of 692 papers across five thematic domains.(b)Annual publication volume by domain (2015–2026). The post-2022 s...
2010
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.