REVIEW 3 major objections 5 minor 25 references
Popular LLMs Amplify Race and Gender Disparities in Human Mobility
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Popular LLMs predict women fewer career-related places and Black and Hispanic individuals far fewer wealth-related places than White individuals, and these predicted gaps exceed real survey gaps.
desk verdict Credible evidence of stereotyped POI predictions, but the 'amplify' claim is unsupported because the NHTS comparison relies on an unvalidated POI-to-work-trip mapping. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a forced-choice POI task with paired categories. Each prompt names one person (or two people in Experiment II) and a randomized list of four points of interest; the model must select one (or two) places. The POI lists are deliberately paired: career-related versus everyday-needs, and wealth-related versus poverty-related. Name-level demographic ratios come from a name-race dataset and a name-gender dataset, feeding logistic regressions that separate name-inferred from explicitly stated demographics. The national travel survey provides the real-world baseline for work travel against which the career-related predictions are judged as reflection or amplification.
What would settle it
Collect real mobility traces with the same eight race-gender groups, classify visited places into the paper's four POI categories, and compare group visit shares with the LLM predictions: if the observed shares match the LLM predictions rather than the national survey, the amplification claim fails, while if they match the survey, it survives.
Extended reading notes
Core claim
The central discovery is that demographic cues in a prompt change an LLM's predicted destinations, and the changes are systematically unequal. In the wealth-versus-poverty comparisons, when race is not mentioned, GPT-4o assigns wealth-related places to all groups at similar high rates; once race is specified, Black and Hispanic individuals' wealth-related shares collapse to near zero in pairwise comparisons, while White individuals' shares remain high. In the career-versus-everyday comparisons, women of every racial group are less likely than men of that group to receive career-related POIs, and the gender gaps are much larger than the gaps found in work-related travel in the 2022 national travel survey. The authors call this amplification: the models do not simply mirror aggregate human behavior, they exaggerate its demographic structure, associating minority men with career places and poverty places in ways the survey does not show.
Load-bearing premise
The amplification claim rests on treating the four career-related POI labels as measuring work-related travel; if those labels are not what people visit on work trips, the model-vs-survey gap measures category mismatch rather than bias amplification.
Editorial extensions
If this is right
- Any downstream system using these models for travel recommendations or urban analytics would inherit a systematic skew: women get fewer career-related destinations, and Black and Hispanic users get fewer wealthy destinations, even when the model is not told their demographics but only their name.
- Bias audits that only measure refusal rates are insufficient: models with high refusal rates still amplify disparities in the answers they do return.
- The reflection-versus-amplification comparison against a national travel survey provides a quantitative test that can be rerun for other models and other POI categories to track fairness over time.
- The paper's results imply that demographic labels are a strong control variable in LLM mobility tasks, so applications that intend to be neutral should either avoid such labels or otherwise correct for their effect.
Reading between the lines
- Not tested in the paper: the four career-related POIs include 'industry conference center' and 'career consultation center,' which are not routine workplaces; a human rating study of whether these POIs represent work trips could change the size of the reported amplification.
- The same prompt design could be extended to other protected attributes such as age, disability, or sexual orientation, or to other behavior categories such as health-care visits and school locations; the paper's framing suggests these are likely to show similar disparities.
- Because the race-specified probes use the same first name with different demographic labels, the models' responses appear label-driven rather than name-driven; this could enable a cheap mitigation test by prepending neutral statistics or counter-stereotypical context before the choice prompt.
- A direct behavioral validation with GPS or cell-phone traces on the exact POI categories would either confirm or overturn the amplification claim; this is the natural next experiment.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper asks whether commercial LLMs (GPT-4o, Gemini-1.5-pro, Claude-3.5-sonnet) encode race and gender bias when predicting human mobility. In Experiment I, the authors prompt the models to choose, from a small fixed set of points of interest (POIs), the place a named individual most likely visited, with or without explicit race/gender labels. They report that minority groups, especially Black and Hispanic individuals, are less often assigned wealth-related POIs and more often assigned poverty-related POIs, while women are less often assigned career-related POIs than men. They compare the career-related POI shares to work-trip shares from NHTS 2022 and conclude that the models do not merely reflect but amplify real-world disparities. Experiment II repeats the comparison in a paired-assignment design, and logistic regressions quantify name-based and label-based effects. The paper's descriptive claim—that LLM outputs are demographically skewed in these ways—is credible, but the stronger 'amplification' conclusion rests on an unvalidated mapping between the experiment's career POIs and NHTS work trips, and on analyses that exclude high, group-correlated refusal rates without reweighting.
Significance. If the descriptive findings hold, they are significant: LLMs are already proposed for travel planning, urban analytics, and mobility simulation, and the demonstration that three major commercial models produce strongly stereotyped predictions about where people go is a valuable fairness result. The paper has genuine strengths: it tests multiple independent model families through their APIs, anchors the bias question against an external survey (NHTS 2022), provides a name-level logistic regression that separates name cues from explicit labels, and reports refusal rates rather than hiding them. These features make the core observation reproducible and worth publishing. However, the headline interpretation—that LLMs 'amplify' rather than merely reflect or distort observed disparities—is currently supported only by a direct comparison of two non-equivalent quantities: a forced-choice share over four hand-picked career POIs and an NHTS trip-purpose share. The paper also lacks uncertainty quantification.
major comments (3)
- [Comparison with Survey Data (Table 1)] The central 'amplify' claim compares LLM shares of career-related POIs with NHTS work-trip shares, but this assumes that the four career POIs in Table S1 (Industry conference center, Employment center, Professional training center, Career consultation center) operationalize work-related travel. Professional training centers and career consultation centers are not workplaces, and the LLM task is a forced choice among four listed POIs, so the resulting category share is not a trip-purpose proportion. No validation of the POI list against actual workplace destinations is provided. As written, the comparison in Table 1 establishes only that the LLM outputs differ from the NHTS benchmark; it does not establish that the model amplifies the real-world disparity. Please either validate the mapping (for example, against workplace visit frequencies in a mobility or location-based services dataset) or reframe the 'amplification' statements in the Abstract, Results, and Discussion as descriptive bias relative to the survey, with the mapping limitation stated explicitly.
- [Methods (last paragraph) and Table 2] Refusal rates are large and group-dependent—Gemini rejects 87.5% of Black wealth-related queries, and Claude rejects 37.5%—yet the Methods state that refusals were excluded from the reported analyses without any reweighting or sensitivity analysis. If refusal propensity is correlated with demographic group and POI category, the non-refused responses are a selected sample, and the extreme estimates in Figure 2 (such as Gemini's wealth-related coefficient below -6 for Black) could reflect differential refusal rather than bias in answered queries. The paper should report refusal-adjusted estimates or bounds (for example, worst-case reallocation of refusals), or explicitly state and defend the assumption that refusals are ignorable. As it stands, this is a load-bearing gap in the support for the quantitative disparity claims, not a minor caveat.
- [Figures 1, 2, and 4; Logistic Regression Analysis] No uncertainty quantification is reported for any of the headline numbers. The paper reports exact point estimates for career-related and wealth-related shares (e.g., 12.2% vs. 2% for White males and females in Figure 1b; 48.0% vs. 0.5% for Black males in Figure 4b) without confidence intervals, bootstrap intervals, repeated API runs, or temperature settings. The logistic regressions in Figure 2 also lack coefficient standard errors despite the text describing some variables as statistically insignificant. Because the choice sets are randomized and the names are sampled, the stability of the cross-group rankings and the extreme coefficients cannot be assessed. At minimum, the authors should supply repeated-run or bootstrap intervals for the main tables and figures, or clearly state the single-run design and its implications for the strength of the claims.
minor comments (5)
- [Table S2] The 'Black Male' list of representative names includes 'Gwendolyn', which is a female name; if this list was used to create male subgroup prompts in Experiment II, the gender comparison for the Black subgroup is contaminated. Please verify the name lists and correct this entry.
- [Amplified Gender and Race Disparities (Figure 4)] The caption says the figure uses GPT-4o, but the surrounding text describes the results as if they apply to LLMs generally; please clarify which model and which experiment each panel reports, and state where the corresponding results for Gemini and Claude appear.
- [Methods, Logistic Regression Analysis] The logistic regression section does not specify the model equation, the handling of multiple observations per name, the standard error calculation, or the threshold used to mark coefficients statistically insignificant in Figure 2; please add this information or point to a supplement.
- [Methods, large language model employment] The description of output extraction as 'simple natural language processing techniques' is vague; please specify the JSON parsing and retry procedure, including how malformed or partial outputs were handled, since refusal and parse failures both affect the effective sample.
- [Reference [20]] The NHTS citation is incompletely formatted ('Federal, H.: Administration'); please supply the full report title, year, and URL or DOI.
Circularity Check
No significant circularity; the amplification claim is anchored to an external NHTS benchmark.
full rationale
The paper's central claim that LLMs amplify, rather than merely reflect, race and gender disparities is supported by comparing LLM-generated POI choice shares to the 2022 National Household Travel Survey (NHTS), an external data source. The paper states: 'we compare the predictions to the 2022 National Household Travel Survey (NHTS) in the U.S. [20] shown in Table 1.' No model output or predicted disparity is defined in terms of a fitted parameter, and no equation makes a prediction equivalent to its input by construction. The logistic regression coefficients are descriptive summaries of the LLM outputs, not fitted quantities used to generate the predictions. The name-based race and gender variables come from external datasets (Rosenman et al., UCI Gender by Name), not from the model's own outputs. There are no load-bearing self-citations: the cited references are model API documentation and prior bias literature, not prior work by the same authors invoked to justify the central premise. The potential weakness that the four career-related POIs may not perfectly operationalize NHTS work trips is a construct-validity concern, not circularity, because the mapping is explicit, externally checkable, and not equivalent by definition. Similarly, the exclusion of refused responses is a sample-selection issue, not a circular derivation. The derivation chain is therefore self-contained against external benchmarks, and no circular step is present.
Assumptions & free parameters
assumptions (4)
- domain assumption Forced-choice POI selections measure human mobility behavior and its biases.
- domain assumption NHTS work-travel proportions are a valid baseline for the career-related POI category.
- domain assumption Name-based racial and gender composition data (Rosenman et al.; UCI Gender by Name) accurately represent the demographic cues the models use.
- domain assumption Excluding rejected responses does not bias the reported comparisons.
Cite this review
Pith. "Pith review of Popular LLMs Amplify Race and Gender Disparities in Human Mobility." pith.science (2026). https://pith.science/paper/BIN7YDQJ
@misc{pith2026241114469,
author = {Pith},
title = {Pith review of: Popular LLMs Amplify Race and Gender Disparities in Human Mobility},
year = {2026},
howpublished = {\url{https://pith.science/paper/BIN7YDQJ}},
note = {Machine review of arXiv:2411.14469}
}
read the original abstract
As large language models (LLMs) are increasingly applied in areas influencing societal outcomes, it is critical to understand their tendency to perpetuate and amplify biases. This study investigates whether LLMs exhibit biases in predicting human mobility -- a fundamental human behavior -- based on race and gender. Using three prominent LLMs -- GPT-4, Gemini, and Claude -- we analyzed their predictions of visitations to points of interest (POIs) for individuals, relying on prompts that included names with and without explicit demographic details. We find that LLMs frequently reflect and amplify existing societal biases. Specifically, predictions for minority groups were disproportionately skewed, with these individuals being significantly less likely to be associated with wealth-related points of interest (POIs). Gender biases were also evident, as female individuals were consistently linked to fewer career-related POIs compared to their male counterparts. These biased associations suggest that LLMs not only mirror but also exacerbate societal stereotypes, particularly in contexts involving race and gender.
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:2103.11943 (2021)
Koroteev, M.V.: Bert: a review of applications in natural language processing and understanding. arXiv preprint arXiv:2103.11943 (2021)
arXiv 2021
-
[2]
arXiv preprint arXiv:2303.08774 (2023)
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)
arXiv 2023
-
[3]
arXiv preprint arXiv:2107.03374 (2021)
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H.P.D.O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al.: Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)
arXiv 2021
-
[4]
arXiv preprint arXiv:2310.05421 (2023)
Pandya, K., Holia, M.: Automating customer service using langchain: Building custom open-source gpt chatbot for organizations. arXiv preprint arXiv:2310.05421 (2023)
arXiv 2023
-
[5]
Nature Human Behaviour 7(9), 1526–1541 (2023) 14
Webb, T., Holyoak, K.J., Lu, H.: Emergent analogical reasoning in large language models. Nature Human Behaviour 7(9), 1526–1541 (2023) 14
work page 2023
-
[6]
Nature Human Behaviour, 1–11 (2024)
Strachan, J.W., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., Saxena, K., Rufo, A., Panzeri, S., Manzi, G., et al.: Testing theory of mind in large language models and humans. Nature Human Behaviour, 1–11 (2024)
work page 2024
-
[7]
Nature Computational Science 3(10), 833–838 (2023)
Hagendorff, T., Fabi, S., Kosinski, M.: Human-like intuitive behavior and reason- ing biases emerged in large language models but disappeared in chatgpt. Nature Computational Science 3(10), 833–838 (2023)
work page 2023
-
[8]
Nature Computational Science 3(9), 748–762 (2023)
Zheng, Y., Lin, Y., Zhao, L., Wu, T., Jin, D., Li, Y.: Spatial planning of urban communities via deep reinforcement learning. Nature Computational Science 3(9), 748–762 (2023)
work page 2023
Show all 25 references
-
[9]
Nature communications 12(1), 6576 (2021)
Simini, F., Barlacchi, G., Luca, M., Pappalardo, L.: A deep gravity model for mobility flows generation. Nature communications 12(1), 6576 (2021)
2021
-
[10]
arXiv preprint arXiv:2405.11715 (2024)
Liu, Y., Kuai, C., Ma, H., Liao, X., He, B.Y., Ma, J.: Semantic trajectory data mining with llm-informed poi classification. arXiv preprint arXiv:2405.11715 (2024)
2024 arXiv
-
[11]
IEEE Transactions on Intelligent Vehicles (2024)
Lan, Z., Liu, L., Fan, B., Lv, Y., Ren, Y., Cui, Z.: Traj-llm: A new exploration for empowering trajectory prediction with pre-trained large language models. IEEE Transactions on Intelligent Vehicles (2024)
2024
-
[12]
arXiv preprint arXiv:2308.15197 (2023)
Wang, X., Fang, M., Zeng, Z., Cheng, T.: Where would i go next? large language models as human mobility predictors. arXiv preprint arXiv:2308.15197 (2023)
2023 arXiv
-
[13]
arXiv preprint arXiv:2407.18932 (2024)
Li, X., Huang, F., Lv, J., Xiao, Z., Li, G., Yue, Y.: Be more real: Travel diary gen- eration using llm agents and individual profiles. arXiv preprint arXiv:2407.18932 (2024)
2024 arXiv
-
[14]
arXiv preprint arXiv:2409.03937 (2024)
Yu, C., Xie, X., Huang, Y., Qiu, C.: Harnessing llms for cross-city od flow prediction. arXiv preprint arXiv:2409.03937 (2024)
2024 arXiv
-
[15]
arXiv preprint arXiv:2409.00063 (2024)
Bhandari, P., Anastasopoulos, A., Pfoser, D.: Urban mobility assessment using llms. arXiv preprint arXiv:2409.00063 (2024)
2024 arXiv
-
[16]
arXiv preprint arXiv:1909.01326 (2019)
Sheng, E., Chang, K.-W., Natarajan, P., Peng, N.: The woman worked as a babysitter: On biases in language generation. arXiv preprint arXiv:1909.01326 (2019)
2019 arXiv
-
[17]
610–623 (2021)
Bender, E.M., Gebru, T., McMillan-Major, A., Shmitchell, S.: On the dangers of stochastic parrots: Can language models be too big? In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 610–623 (2021)
2021
-
[18]
arXiv preprint arXiv:2005.14050 (2020) 15
Blodgett, S.L., Barocas, S., Daum´ e III, H., Wallach, H.: Language (technology) is power: A critical survey of” bias” in nlp. arXiv preprint arXiv:2005.14050 (2020) 15
2020 arXiv
-
[19]
arXiv preprint arXiv:2402.04105 (2024)
Bai, X., Wang, A., Sucholutsky, I., Griffiths, T.L.: Measuring implicit bias in explicitly unbiased large language models. arXiv preprint arXiv:2402.04105 (2024)
2024 arXiv
-
[20]
2022 nextgen national household travel survey core data
Federal, H.: Administration. 2022 nextgen national household travel survey core data. US Department of Transportation: Washington, DC, USA (2022)
2022
-
[21]
Version: GPT-4o-2024-08-06 (2024)
OpenAI: GPT-4 API. Version: GPT-4o-2024-08-06 (2024). https://platform. openai.com/docs/models/gpt-4
2024
-
[22]
Version: Gemini-1.5-pro (2024)
DeepMind: Gemini 1.5 API. Version: Gemini-1.5-pro (2024). https://www. deepmind.com/gemini
2024
-
[23]
Version: Claude-3-5-sonnet-20240620 (2024)
Anthropic: Claude 3.5 API. Version: Claude-3-5-sonnet-20240620 (2024). https: //www.anthropic.com/claude
2024
-
[24]
Scientific Data 10(1), 299 (2023)
Rosenman, E.T., Olivella, S., Imai, K.: Race and ethnicity data for first, middle, and surnames. Scientific Data 10(1), 299 (2023)
2023
-
[25]
DOI: https://doi.org/10.24432/C55G7X (2020) 16 Appendix A Specification of experiments In the experiments, the POI sets for four categories are presented as follows
UCI Machine Learning Repository: Gender by Name. DOI: https://doi.org/10.24432/C55G7X (2020) 16 Appendix A Specification of experiments In the experiments, the POI sets for four categories are presented as follows. Career-related POIs Industry conference center Employment cent...
2020 doi
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.