NLP needs Diversity outside of 'Diversity'
Pith reviewed 2026-05-10 12:05 UTC · model grok-4.3
The pith
Diversity progress in NLP concentrates in fairness areas because barriers push marginalized researchers out of other subfields.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper establishes that diversity progress in NLP is disproportionately concentrated on fairness areas as the result of incentives, biases, and barriers that together disenfranchise marginalized researchers in non-fairness fields or move them into fairness-related work. This is substantiated through an investigation of the demographics of NLP researchers by subfield, which in turn supports recommendations for breaking feedback loops that reinforce disparities and for addressing geographical and linguistic barriers to participation.
What carries the argument
Demographic investigation of NLP researchers by subfield, used to identify feedback loops and geographical plus linguistic barriers as the mechanisms concentrating diversity efforts in fairness.
If this is right
- Breaking down feedback loops will allow marginalized researchers to contribute to non-fairness areas of NLP without being redirected.
- Addressing geographical and linguistic barriers will increase equitable participation across all NLP subfields.
- All areas within NLP can become more inclusive and equitable once the identified mechanisms are targeted.
- Diversity efforts will extend beyond fairness topics when the barriers that reinforce concentration are removed.
Where Pith is reading between the lines
- The same concentration pattern may appear in other areas of machine learning, suggesting parallel demographic studies could reveal comparable dynamics.
- Removing the barriers could surface new research questions in non-fairness subfields that draw on perspectives currently underrepresented.
- Tracking subfield participation rates after conference and funding policies change would provide a direct test of whether the recommended actions shift the observed demographics.
Load-bearing premise
The observed concentration of marginalized researchers in fairness subfields results primarily from disenfranchising incentives, biases, and barriers rather than from differences in personal interests or in resources available across subfields.
What would settle it
A survey asking marginalized NLP researchers about their subfield preferences and the pressures they face when choosing research topics, with results showing whether choices align more with external barriers or with independent interests.
Figures
read the original abstract
This position paper argues that recent progress with diversity in NLP is disproportionately concentrated on a small number of areas surrounding fairness. We further argue that this is the result of a number of incentives, biases, and barriers which come together to disenfranchise marginalized researchers in non-fairness fields, or to move them into fairness-related fields. We substantiate our claims with an investigation into the demographics of NLP researchers by subfield, using our research to support a number of recommendations for ensuring that all areas within NLP can become more inclusive and equitable. In particular, we highlight the importance of breaking down feedback loops that reinforce disparities, and the need to address geographical and linguistic barriers that hinder participation in NLP research.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This position paper claims that recent diversity progress in NLP is disproportionately concentrated in fairness-related areas, resulting from incentives, biases, and barriers that disenfranchise marginalized researchers in non-fairness subfields or push them toward fairness work. It substantiates the claims via a demographic investigation of NLP researchers by subfield and offers recommendations focused on breaking reinforcing feedback loops and addressing geographical and linguistic barriers to broaden inclusivity across all NLP areas.
Significance. If the causal interpretation holds, the paper would usefully highlight risks of narrow diversity efforts and provide actionable recommendations for field-wide equity. It correctly flags the potential for self-reinforcing disparities and gives credit to existing fairness work while advocating expansion. The position is timely for NLP venues, though its impact hinges on strengthening the evidential link between observed demographics and the proposed mechanisms.
major comments (2)
- [Abstract] Abstract: The claim that the demographic investigation substantiates causal links to 'incentives, biases, and barriers' which 'disenfranchise' researchers is load-bearing for the central argument, yet the abstract (and apparent methods) provides no details on data sources, sample sizes, statistical controls, or tests against alternatives such as subfield popularity, entry costs, or self-selection by interest; without these, the data yield correlation but cannot adjudicate the primary-cause interpretation.
- [Demographic investigation] Demographic investigation section: Interpreting subfield concentration as evidence of disenfranchisement via feedback loops risks circularity, as the analysis appears shaped by the initial framing without reported external benchmarks, regression controls for confounds, or explicit comparison to falsifiable alternative models (e.g., interest surveys or resource availability by subfield).
minor comments (2)
- Define 'fairness-related fields' and the subfield categorization scheme explicitly, including how borderline areas (e.g., ethics vs. core NLP) were assigned.
- Add a limitations subsection discussing potential biases in the demographic data collection and how they might affect the concentration findings.
Simulated Author's Rebuttal
We thank the referee for their careful reading and constructive suggestions. We agree that greater transparency around the supporting analysis and explicit discussion of interpretive limits will strengthen the paper, and we will revise accordingly while preserving the position paper's focus on observed patterns and recommendations.
read point-by-point responses
-
Referee: [Abstract] Abstract: The claim that the demographic investigation substantiates causal links to 'incentives, biases, and barriers' which 'disenfranchise' researchers is load-bearing for the central argument, yet the abstract (and apparent methods) provides no details on data sources, sample sizes, statistical controls, or tests against alternatives such as subfield popularity, entry costs, or self-selection by interest; without these, the data yield correlation but cannot adjudicate the primary-cause interpretation.
Authors: We accept that the abstract should more clearly frame the evidential role of the demographic analysis. The paper does not assert that the data alone establish primary causation; the concentration patterns are presented as one piece of supporting evidence alongside well-documented field incentives (e.g., venue and funding priorities). In the revision we will (1) specify the data sources and approximate sample in the abstract, (2) add a short methods paragraph describing the observational approach and its limitations, and (3) explicitly note that causal mechanisms are argued interpretively rather than proven statistically. These changes will distinguish correlation from the broader argument without altering the paper's position. revision: yes
-
Referee: [Demographic investigation] Demographic investigation section: Interpreting subfield concentration as evidence of disenfranchisement via feedback loops risks circularity, as the analysis appears shaped by the initial framing without reported external benchmarks, regression controls for confounds, or explicit comparison to falsifiable alternative models (e.g., interest surveys or resource availability by subfield).
Authors: We agree that the section should guard against circularity. The demographic investigation reports observed participation rates; the feedback-loop framing is offered as a plausible interpretation informed by existing literature on academic incentives, not as a data-derived conclusion. In revision we will add an explicit subsection discussing alternative explanations (self-selection, subfield popularity, entry costs) and reference available external benchmarks on overall NLP subfield distributions where possible. Because the work is observational and draws on existing public data, we cannot introduce new primary data such as interest surveys or controlled regressions; we will therefore state these limits clearly and invite readers to evaluate the interpretive weight independently. revision: partial
- New primary data collection (e.g., interest surveys or resource-availability metrics by subfield) to enable formal statistical tests against alternative models; such work lies outside the scope of a position-paper revision.
Circularity Check
No circularity detected; position paper presents independent demographic observations and interpretive recommendations.
full rationale
The paper's core argument—that diversity efforts concentrate in fairness subfields due to incentives, biases, and barriers—is substantiated by a demographic investigation of NLP researchers by subfield. This is an empirical presentation of data followed by recommendations, not a derivation chain. No equations, parameter fittings, self-definitional constructs, uniqueness theorems, or self-citations reduce any claim to its own inputs by construction. The interpretation linking demographics to disenfranchisement is an open causal claim open to alternative explanations, but does not meet the criteria for circularity under the specified patterns.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Observed demographic distributions across NLP subfields reflect the effects of incentives, biases, and barriers on marginalized researchers
Reference graph
Works this paper leans on
-
[1]
John Fitzgerald, Sanna Ojanperä, and Neave O’Clery
V oices of her: Analyzing gender differences in the ai publication world. John Fitzgerald, Sanna Ojanperä, and Neave O’Clery
-
[2]
Vagrant Gautam, Arjun Subramonian, Anne Lauscher, and Os Keyes
Is academia becoming more localised? the growth of regional knowledge networks within in- ternational research collaboration.Applied Network Science, 6(1):1–27. Vagrant Gautam, Arjun Subramonian, Anne Lauscher, and Os Keyes. 2024. Stop! in the name of flaws: Disentangling personal names and sociodemographic attributes in NLP. InProceedings of the 5th Work...
work page 2024
-
[3]
Dirk Hovy and Shrimai Prabhumoye
Diversity, equity, and inclusion in research teams: The good, the bad, and the ugly.Race and Justice, 12(3):505–530. Dirk Hovy and Shrimai Prabhumoye. 2021. Five sources of bias in natural language processing.Lan- guage and linguistics compass, 15(8):e12432. How Does the Search Work? 2025. How does the search work? https://aclanthology.org/ search. Access...
work page 2021
-
[4]
Evaluating the diversity, equity, and inclu- sion of NLP technology: A case study for Indian languages. InFindings of the Association for Compu- tational Linguistics: EACL 2023, pages 1763–1777, Dubrovnik, Croatia. Association for Computational Linguistics. JA Knutsen and S Presser. 2010. Question and question- naire design.Handbook of survey research, pa...
-
[5]
An overview of recent approaches to enable diversity in large language models through align- ing with human perspectives. InProceedings of the 3rd Workshop on Perspectivist Approaches to NLP (NLPerspectives) @ LREC-COLING 2024, pages 49– 55, Torino, Italia. ELRA and ICCL. Donna J Nelson and Diana C Rogers. 2003.A national analysis of diversity in science ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.