Pith. sign in

REVIEW 3 major objections 1 minor 1 cited by

Analysis of 4 million applications screened by the same algorithmic vendor shows racial disparities in adverse outcomes and higher-than-chance homogeneity in individual rejections.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-06-29 15:07 UTC pith:WKHIYZL6

load-bearing objection Large dataset from one hiring vendor shows measurable homogeneity in rejections and racial disparities, but the link to monoculture needs tighter controls on job and applicant differences. the 3 major comments →

arxiv 2605.27371 v1 pith:WKHIYZL6 submitted 2026-05-26 cs.CY cs.AI

Algorithmic Monocultures in Hiring

Rishi Bommasani , Sarah H. Bana , Kathleen A. Creel , Dan Jurafsky , Percy Liang This is my paper
classification cs.CY cs.AI
keywords applicantsapplicationspositionssamealgorithmsoutcomesalgorithmicapply
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The authors collected a dataset of 3 million people submitting 4 million job applications, all processed by hiring algorithms from a single company. They measured how often applications from Asian and Black candidates went to jobs where the algorithm produced outcomes that would count as adverse impact under U.S. employment rules, finding rates of 14.74 percent and 25.87 percent respectively. They also tracked what happened to the same person across multiple applications and found that 4 percent of people who applied to ten positions were rejected from every one, a pattern more common than random chance would predict. Because the algorithms are deterministic, the authors could simulate what would have happened if every applicant had applied to every job. The simulation suggests that applicants must submit many applications before any reach a human reviewer. The work treats the shared vendor as the source of both group-level disparities and individual-level uniformity in outcomes.

Core claim

Of all applications submitted by Asian and Black applicants, 14.74% and 25.87% are submitted to positions that adversely impact Asian and Black applicants, respectively, according to U.S. employment discrimination standards. Individuals also receive homogeneous outcomes: 4% of all applicants who apply to 10 positions are recommended for rejection from all positions, a rate higher than expected by chance.

Load-bearing premise

That the observed homogeneity and disparities are caused by the shared algorithmic vendor rather than by applicant pool composition, job posting differences, or other unmeasured factors; the abstract states the dataset consists of applications screened by algorithms built by the same vendor but does not detail how vendor identity was verified or how confounding variables were controlled.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 1 minor

Summary. The paper claims that algorithmic monoculture—multiple employers using hiring algorithms from the same vendor—produces homogeneous rejection outcomes for the same individuals and racial disparities in screening. Analyzing a dataset of 3 million applicants submitting 4 million applications, all screened by algorithms from one vendor, it reports that 14.74% of applications from Asian applicants and 25.87% from Black applicants go to positions that adversely impact those groups per U.S. employment discrimination standards. It further finds that 4% of applicants applying to 10 positions are rejected from all of them (higher than chance), and uses deterministic simulation to show that wide application is needed to reach human review.

Significance. If the observed homogeneity and disparities can be isolated to the shared vendor after appropriate controls, the result would provide concrete evidence of reduced outcome diversity from concentrated algorithmic providers, with direct relevance to employment law, vendor concentration policy, and fairness auditing. The large-scale observational dataset and exploitation of algorithmic determinism for counterfactual simulation are methodological strengths that enable falsifiable claims about individual-level homogeneity.

major comments (3)
  1. [Dataset description / Methods] Dataset and methods section: the manuscript states that all 4 million applications were screened by algorithms from the same vendor but provides no description of how vendor identity was verified (e.g., model signatures, API provenance, or metadata) across the applications, nor any controls or matching for job posting content, applicant pool composition, or other factors that could produce correlated outcomes even with independent algorithms. This directly undermines the attribution of the 14.74%/25.87% adverse-impact figures and the 4% homogeneity rate to monoculture.
  2. [Results (adverse impact analysis)] Results on adverse impact: the abstract and results report the 14.74% and 25.87% figures for Asian and Black applicants but give no information on the exact adverse-impact ratio thresholds applied, how they were computed from the algorithmic scores, or whether applicant or job characteristics were balanced before calculating the percentages.
  3. [Results (homogeneity analysis)] Homogeneity result: the claim that 4% of applicants applying to 10 positions are rejected from all is presented as higher than expected by chance, yet the manuscript supplies no details on the statistical test used, the null model for chance expectation, or whether applicant/job characteristics were balanced in the comparison.
minor comments (1)
  1. [Abstract] Abstract: the sentence on the simulation result is slightly unclear about the exact counterfactual being generated; rephrasing for precision would help.

Simulated Author's Rebuttal

3 responses · 1 unresolved

We thank the referee for the constructive feedback on our manuscript. The comments highlight areas where additional methodological detail will improve clarity. We respond to each major comment below.

read point-by-point responses

  1. Referee: [Dataset description / Methods] Dataset and methods section: the manuscript states that all 4 million applications were screened by algorithms from the same vendor but provides no description of how vendor identity was verified (e.g., model signatures, API provenance, or metadata) across the applications, nor any controls or matching for job posting content, applicant pool composition, or other factors that could produce correlated outcomes even with independent algorithms. This directly undermines the attribution of the 14.74%/25.87% adverse-impact figures and the 4% homogeneity rate to monoculture.

    Authors: The dataset was obtained directly from the vendor, who confirmed that every application was processed exclusively through their screening algorithms. Confidentiality agreements prevent disclosure of model signatures or API metadata. We will revise the methods section to state the data source and vendor confirmation explicitly. The analysis is observational and captures outcomes under a shared vendor; we will add a limitations paragraph discussing potential confounders and the infeasibility of matching or controls given the data structure. revision: partial

  2. Referee: [Results (adverse impact analysis)] Results on adverse impact: the abstract and results report the 14.74% and 25.87% figures for Asian and Black applicants but give no information on the exact adverse-impact ratio thresholds applied, how they were computed from the algorithmic scores, or whether applicant or job characteristics were balanced before calculating the percentages.

    Authors: The percentages reflect positions where the algorithmic selection rate for the focal group fell below 80% of the highest group's rate, per the Uniform Guidelines on Employee Selection Procedures. Computation used the raw algorithmic scores without balancing applicant or job characteristics, as the figures describe observed real-world outcomes. We will add the precise threshold definition, computation steps, and note on the absence of balancing to the results section. revision: yes

  3. Referee: [Results (homogeneity analysis)] Homogeneity result: the claim that 4% of applicants applying to 10 positions are rejected from all is presented as higher than expected by chance, yet the manuscript supplies no details on the statistical test used, the null model for chance expectation, or whether applicant/job characteristics were balanced in the comparison.

    Authors: The 4% rate is compared against a null model that randomly assigns rejections according to the dataset-wide rejection probability for each applicant. A simulation-based test (or equivalent binomial comparison) was used to evaluate whether the observed all-rejection rate exceeds the null. No balancing of characteristics was applied, to preserve the empirical distribution of applications. We will expand the results section with the full description of the test, null model, and rationale for the comparison. revision: yes

standing simulated objections not resolved
  • Specific model signatures, API provenance, or metadata used to verify the single vendor cannot be disclosed due to confidentiality agreements with the data provider.

Circularity Check

0 steps flagged

No circularity: results are direct observational counts and deterministic extrapolation from the provided dataset.

full rationale

The paper reports raw percentages (14.74%, 25.87%, 4%) and homogeneity rates computed directly from the 4M applications in its novel dataset, all screened by one vendor's algorithms. The simulation step uses the deterministic replicability of those algorithms to generate counterfactual outcomes for wider application, without any fitted parameters, self-definitional equations, or load-bearing self-citations that reduce the reported statistics to their own inputs by construction. Attribution concerns (vendor verification, confounder controls) are methodological but do not create circularity in the derivation chain.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The work is observational and relies on external legal standards for adverse impact rather than new theoretical constructs; no free parameters are introduced to fit the reported numbers, and no new entities are postulated.

axioms (2)
  • domain assumption U.S. employment discrimination standards provide a valid threshold for labeling an outcome as adverse impact
    The percentages 14.74% and 25.87% are defined with reference to these standards; the abstract invokes them without further justification.
  • domain assumption The observed homogeneity rate exceeds what would be expected under independent random assignment of outcomes
    The claim that 4% is higher than chance requires an implicit null model of independence across applications that is not detailed in the abstract.

reviewed 2026-06-29 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Algorithmic Monocultures in Hiring." pith.science (2026). https://pith.science/paper/WKHIYZL6

@misc{pith2026260527371,
  author       = {Pith},
  title        = {Pith review of: Algorithmic Monocultures in Hiring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WKHIYZL6}},
  note         = {Machine review of arXiv:2605.27371}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Many employers screen job applicants with algorithms built by the same few algorithm vendors. We hypothesize that algorithmic monoculture leads to the same individuals and members of the same racial groups facing rejection. We acquire and analyze a novel dataset of 3 million applicants submitting 4 million applications where all the applications are screened by algorithms built by the same vendor. We find clear racial disparities in applicant outcomes. Of all applications submitted by Asian and Black applicants, 14.74% and 25.87% are submitted to positions that adversely impact Asian and Black applicants, respectively, according to U.S. employment discrimination standards. Individuals also receive homogeneous outcomes: 4% of all applicants who apply to 10 positions are recommended for rejection from all positions, a rate higher than expected by chance. To better understand this homogeneity, we leverage the deterministic replicability of hiring algorithms to generate the outcomes applicants would have received if they applied to all positions. We show that applicants would need to apply widely in order to ensure their applications are considered by a human

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. The Hitchhiker's Guide to Monoculture

    cs.CY 2026-07 conditional novelty 6.0

    Code written for Kaggle contests has become substantially more similar in syntax since ChatGPT, converging heavily on seed 42, while the semantic diversity of solution approaches has not declined.

This paper was first reviewed by grok-4.3 on June 29, 2026.