Humble AI in the real-world: the case of algorithmic hiring

Bran Knowles; Christos Varytimidis; Elizabeth Daly; Inge Vejsbjerg; Rahul Nair

REVIEW 3 major objections 5 minor 8 references

Humble AI in the real-world: the case of algorithmic hiring

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Point-score rankings in algorithmic screening are undercut by noise; ranking candidates by expected rank instead substantially recovers true fit and shifts who gets surfaced.

desk verdict Short feasibility study with a near-tautological synthetic validation and an overread real-data comparison; still worth a careful referee. read the letter →

arxiv 2505.20918 v1 pith:DWDRVN5Q submitted 2025-05-27 cs.LG cs.AIcs.HC

Rahul Nair , Inge Vejsbjerg , Elizabeth Daly , Christos Varytimidis , Bran Knowles This is my paper

classification cs.LGcs.AIcs.HC

keywords algorithmichiringuncertaintyquantificationranksetsexpectedentropycandidatescreeninghumbleAIbiasedoverlap

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a standard AI-screening ranking, ordering candidates by a single predicted score, can be arbitrary because the score is noisy, and that making the ranking humble by accounting for that uncertainty produces rankings that are both more reliable and closer to the true order. It operationalizes Humble AI, a stance of scepticism, curiosity, and commitment, through rank sets, entropy estimates, and a recruiter-facing user experience that exposes algorithmic unknowns. In synthetic experiments where the true ranking is known, expected-rank rankings deviate from the truth considerably less than point-score rankings as noise grows. On real job postings and anonymized candidate profiles, the deterministic and uncertainty-aware rankings show very little correspondence, which the authors read as evidence that point-score rankings can surface candidates arbitrarily. A focus group with recruiters and HR specialists found the premise receptive but flagged that uncertainty displays may be misattributed to the candidate rather than the model.

What carries the argument

The central object is the rank set, a probability matrix $P=[p_{ij}]$ where $p_{ij}$ is the probability that candidate $i$ occupies rank $j$. The matrix is estimated by local perturbations: the candidate vector is randomly feature-masked in the style of the explainability method LIME, the black-box scoring function is evaluated on each perturbation, and Monte Carlo draws aggregate into rank probabilities. From $P$ the authors compute the expected rank $\sum_j p_{ij} j$, rank entropy (the entropy of the candidate's rank-probability row), and rank variance. Expected rank supplies the humble ranking, entropy flags candidates whose position is least reliable and may need manual review, and Rank Biased Overlap (RBO) measures agreement, or disagreement, with deterministic rankings.

What would settle it

Use the same hiring platform's scoring function but obtain ground-truth outcomes, such as interview success or later job performance, for screened candidates, then compare how often deterministic point-score rankings and expected-rank rankings recover that ground truth under realistic noise; if deterministic rankings match ground truth at least as often, the claim that point-score rankings are arbitrary is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that the single point score assigned by a screening model is a noisy estimate of true fit, and that ranking candidates by that point score can therefore be arbitrary. The remedy is to replace the point score with the expected rank derived from a probability distribution over ranks: in a synthetic setting with known true ranking and added Gaussian noise, using average rank instead of the raw score considerably reduces deviations from the true ranking as noise increases. On real data from a hiring platform, the correspondence between the deterministic ranking and the uncertainty-aware ranking is very low, with RBO values near zero for several job titles, which the authors take as validating their argument for caution. The paper's position is that humility is not merely a safety value but a performance improvement: accounting for uncertainty has value in performance.

Load-bearing premise

The argument's load-bearing premise is that the noise around a candidate's true score is symmetric and that random feature masking faithfully represents what the model does not know; if real hiring-score noise is skewed or the masking is unfaithful, the expected-rank advantage weakens and disagreement between ranking schemes no longer proves arbitrariness.

Editorial extensions

If this is right

Presenting recruiters with the expected-rank ordering instead of the raw-score ordering should reduce the chance that top-k lists are populated by noise-driven outliers.
Candidates with high rank entropy should be surfaced for manual review rather than silently ranked, because their position is the least trustworthy.
Because the agreement between deterministic and uncertainty-aware rankings is low, adopting the humble view can change which candidates are surfaced, not just reorder the same set.
The synthetic experiments quantify a concrete benefit: as score noise grows, expected-rank rankings deviate from the true ranking considerably less than point-score rankings.
Higher cognitive load from uncertainty displays will require deliberate user-experience design and recruiter training, as the focus group noted.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension would replace symmetric normal noise with skewed, heavy-tailed, or model-specific error distributions; expected-rank gains may persist or shrink, bounding how far the synthetic result generalizes.
If rank distributions are well calibrated, the same machinery could transfer to other consequential ranking tasks, such as loan decisions, medical triage, or scholarship shortlists, where point scores are treated as exact.
The focus group's warning that displayed uncertainty may be misread as a property of the candidate, not the model, suggests an explicit design test: ask users where they attribute the uncertainty after using the prototype.
A stronger real-world validation would correlate rank entropy with downstream outcomes such as interview shortlisting or job performance; if high-entropy candidates are not systematically misranked, the practical value of the entropy signal would need rethinking.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Honest take: this is a short feasibility study, not a validated intervention. The central technical move—treating a black-box screener's score as a random variable, building a rank distribution via LIME-style feature masking, and summarizing it with expected rank and entropy—is reasonable and, as far as I know, not common in hiring screening. The UX prototype and the focus-group feedback are genuine additions, and the authors are candid about the difficulty of validating uncertainty in hiring.

What's good: the rank-set formulation generalizes earlier work by relaxing bounded ranges to full probabilities, and the Monte Carlo estimator is simple and sensible. The paper does not hide its limitations; the Limitations paragraph explicitly says true screening outcomes are unknown. That honesty matters.

What's soft: the synthetic experiment is close to a tautology. If scores are drawn from a normal around the true value, the expected rank across draws will mechanically converge to the true rank, while a single noisy draw will not. So the RBO improvement in Figure 4 mostly re-states the generative assumption. The real-data claim is shakier. Table 1 shows low agreement between the deterministic ranking and the expected-rank ranking, but that doesn't prove deterministic rankings are arbitrary. It just shows the two ranking procedures differ, and the difference is entirely a function of the perturbation family. LIME-style random masking is one arbitrary choice; a different perturbation mechanism would likely produce different expected ranks. Without ground truth, low agreement is not evidence of arbitrariness. The paper's own sentence 'These results validate our central argument' goes beyond what the data can support.

Minor issues: the entropy formula seems mis-indexed (it sums over i instead of j for a candidate's distribution), and the perturbation parameters are underspecified. No code or data are released, so reproducibility is low.

Who it's for: people working on uncertainty visualization in high-stakes ranking, and HCI folks designing for AI skepticism. They'll get a useful pointer and a clear example of the design problems. The evidentiary claims need to be rewritten as hypotheses, not conclusions.

Recommendation: I'd send it to peer review, not desk reject. The problem is important, the approach has promise, and the authors are honest about limits. But I'd expect the reviewers to push for a simpler baseline in the synthetic study, a reframing of the real-data 'validation' as a demonstration of feasibility, and either code release or fully specified parameters.

Referee Report

3 major / 5 minor

Summary. The paper applies Humble AI principles to algorithmic candidate screening in hiring. It proposes representing the output of a black-box scoring function as rank sets via Monte Carlo perturbation (LIME-style random feature masking) and ranking candidates by expected rank, supplemented by per-candidate entropy. It reports a synthetic experiment (Sec. 2.2) comparing deterministic and expected-rank rankings under added Gaussian noise with true ranking known, and a real-world demonstration on a hiring platform with anonymized candidate profiles for multiple job titles, in which agreement between deterministic and uncertainty-aware rankings is low (Table 1). It also describes a recruiter focus group and a UI prototype that surfaces rank uncertainty and high-entropy candidates. The authors conclude that point-score rankings can be arbitrary and that uncertainty-aware rankings improve recovery of true rankings.

Significance. If the empirical claims were established, the work would contribute a practical instantiation of Humble AI and a cautionary result for hiring platforms: point estimates of candidate scores can mis-rank candidates, and exposing uncertainty could change screening decisions. The paper's strengths are the rank-set formalism, the black-box perturbation approach, and early stakeholder feedback suggesting receptivity. However, the central validation is weak: the synthetic experiment encodes the conclusion in the noise model, and the real-data disagreement is not connected to a ground truth or to a validated perturbation distribution. The paper itself acknowledges this limitation in the Limitations paragraph, yet later states that the results 'validate our central argument.' The contribution is therefore more of an exploratory design and position work than an empirically validated technical claim; it could become a solid case study if the claims are reframed and the perturbation method is validated or compared with alternatives.

major comments (3)

[Section 2.2, Figure 4] The synthetic validation is close to tautological. The text states that 'scores generated are drawn from a normal distribution around the true value'; under this generative model, averaging over repeated perturbations (expected rank) mechanically re-centers the noisy scores around the true value, so the expected-rank ranking will recover the true ranking better than a single noisy draw. The RBO improvement in Figure 4 is therefore a property of the assumed symmetric noise model, not evidence about real hiring-score noise. The paper should either present the synthetic result as a sanity check of the estimator, or test the robustness of expected-rank ranking under asymmetric, heteroscedastic, or skewed noise and under misspecification of the perturbation distribution.
[Section 3, Table 1 and Limitations] The low Jaccard/RBO agreement between the deterministic ranking and the expected-rank ranking is not sufficient to show that point-score rankings are arbitrary. Expected ranks are computed from a specific perturbation distribution (LIME-style random feature masking), and no evidence is given that this distribution faithfully represents the model's ignorance about each candidate; other plausible perturbation families could yield materially different expected ranks. The Limitations paragraph concedes that the true screening outcome is not known, yet the authors state that the results 'validate our central argument for caution in deployment.' To support the arbitrariness claim, the paper would need an external validation (e.g., downstream outcomes, expert judgments, or comparison of multiple perturbation families), or the wording would need to be softened to a design motivation rather than an empirical validation.
[Section 2, rank-set estimation] The rank-set procedure is not reproducible as described. The paper fixes a rank probability threshold (0.01) but does not report the number of Monte Carlo draws, the fraction of features masked per perturbation, the number of perturbations per candidate, or the representation of candidate profiles used for masking. These choices can strongly affect the estimated rank probabilities and hence the agreement measures in Table 1 and Figure 4; a sensitivity analysis or at least a full specification is needed before the empirical results can be assessed.

minor comments (5)

[Section 2, entropy definition] The entropy expression should sum over ranks j, not candidates i; as written, the formula summing over i in n does not define a per-candidate entropy, so it should be corrected to sum over j.
[Table 1 and appendix] The caption lists 'Similarity' as Jaccard Similarity and then reports RBO, but it should clarify which top-k sets are being compared (e.g., top-50 candidates) and which RBO parameter p is used, so that the comparison is interpretable.
[Figure 4] The figure axes and legend do not state the number of candidates, the noise range, the number of trials, or the RBO parameter; adding these details and error bars would make the comparison more informative.
[Section 4] The section is titled 'User studies' but it reports a single half-day focus group with 30 stakeholders; renaming it 'Focus group' or 'Preliminary stakeholder workshop' would better match the evidence presented.
[Section 2, rank variance] Rank variance is mentioned as one of the quantities of interest, but no formula is given; adding a definition would make the method self-contained.

Circularity Check

1 steps flagged · score 6.0 of 10

Synthetic expected-rank 'improvement' is forced by the zero-mean noise definition; real-data agreement is unvalidated but not circular.

self definitional [Section 2.2 (A synthetic case), Figure 4]
"For a set of candidates for whom a true ranking is assumed to be known, their scoring function is perturbed with a known amount of noise. The scores generated are drawn from a normal distribution around the true value. ... When one includes the distribution of ranks and uses average rank instead, the deviations from true rankings are considerably reduced as seen in Figure 4b."

The experiment defines the 'true value' as the mean of the normal distribution from which noisy scores are drawn. The expected-rank ranking is the ranking of those same draws' means (via the Monte Carlo rank probabilities). In the large-sample limit the expected rank orders candidates exactly by the true scores, so the RBO improvement over a single noisy draw is guaranteed by the zero-mean noise assumption. The result is therefore equivalent to the experiment's generative input, not evidence that LIME-style perturbations in real hiring data capture true suitability. The paper's later claim that this validates the central argument inherits the same construction.

full rationale

The synthetic experiment (Sec 2.2) is the only place the paper claims a quantitative performance gain from uncertainty-aware ranking; however that gain is baked into the generative model. The real-data comparison (Sec 3, Table 1) reports low agreement between deterministic and uncertainty-aware rankings, which is an empirical observation and not circular; the contested step is the interpretive leap that low agreement proves deterministic rankings are arbitrary. The paper's own Limitations paragraph concedes that true screening outcomes are unknown and that only the synthetic case offers validation, so the real-data leg is better classified as unsupported rather than circular. The Humble AI framework is cited from the authors' prior work [5], but the technical definitions (rank sets, entropy, perturbation sampling) are independently specified and no mathematical conclusion is imported from that citation. Thus the only circularity is the self-definitional synthetic validation, which is load-bearing for the 'performance' claim but not for the whole paper, giving partial circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or causal entities; rank sets and entropy are representations of existing scores. The main external burden is the assumed faithfulness of local perturbation distributions and the normal-noise model used for synthetic validation.

free parameters (5)

Rank probability threshold = 0.01
Probabilities below 0.01 are dropped when constructing rank sets; chosen for the experiments with no sensitivity analysis.
Candidate pool size = 1000
Assumed for the real-data screening scenario in Section 3, not tied to measured pool sizes.
Top-k candidates shown = 50
Assumption in Section 3 for the recruiter screening scenario.
Synthetic noise variance range = Varied in experiments
Controls difficulty in Section 2.2; the RBO improvement depends on this noise model.
Perturbation count and masking fraction
Required by the LIME-style local perturbation in Section 2 but not specified; rank probabilities depend on these settings.

assumptions (5)

domain assumption The scoring function is available only as a black box for inference.
Section 2 defines the problem with black-box access; the perturbation method is designed around this constraint.
domain assumption Random feature masking yields a meaningful empirical score distribution for each candidate.
Central to rank-set estimation in Section 2; not validated against any real ground-truth score distribution.
domain assumption Entropy of rank probabilities indicates which candidates need additional manual review.
Section 2.1 uses high-entropy candidates as needing review; no evidence links entropy to recruiter value or hiring quality.
domain assumption Higher scores indicate greater suitability for the job.
Stated at the start of Section 2 and required for the ranking interpretation.
ad hoc to paper Synthetic scores are drawn from a normal distribution around the true value.
Section 2.2 constructs the synthetic experiment this way, which makes expected-rank improvement a near-mechanical consequence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Humble AI in the real-world: the case of algorithmic hiring." pith.science (2026). https://pith.science/paper/DWDRVN5Q

@misc{pith2026250520918,
  author       = {Pith},
  title        = {Pith review of: Humble AI in the real-world: the case of algorithmic hiring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DWDRVN5Q}},
  note         = {Machine review of arXiv:2505.20918}
}

read the original abstract

Humble AI (Knowles et al., 2023) argues for cautiousness in AI development and deployments through scepticism (accounting for limitations of statistical learning), curiosity (accounting for unexpected outcomes), and commitment (accounting for multifaceted values beyond performance). We present a real-world case study for humble AI in the domain of algorithmic hiring. Specifically, we evaluate virtual screening algorithms in a widely used hiring platform that matches candidates to job openings. There are several challenges in misrecognition and stereotyping in such contexts that are difficult to assess through standard fairness and trust frameworks; e.g., someone with a non-traditional background is less likely to rank highly. We demonstrate technical feasibility of how humble AI principles can be translated to practice through uncertainty quantification of ranks, entropy estimates, and a user experience that highlights algorithmic unknowns. We describe preliminary discussions with focus groups made up of recruiters. Future user studies seek to evaluate whether the higher cognitive load of a humble AI system fosters a climate of trust in its outcomes.

Figures

Figures reproduced from arXiv: 2505.20918 by the authors.

**Figure 2.** User experience for job specifications (a) Candidate matches (deterministic) (b) Candidate matches with humble AI principles [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** User experience for candidate screening Three quantities are of interest are computed: (a) the expected rank of the 𝑖-th candidate, i.e. Í 𝑗 ∈𝑛 𝑝𝑖𝑗 𝑗, (b) the entropy for each candidate 𝑖, i.e. Í 𝑖∈𝑛 𝑝𝑖𝑗 log 𝑝𝑖𝑗 , and (c) rank variance. We additionally experimented with determining the most likely ranking based on linear programming. However this turned out to be uninformative when rank variance is high. 2.1 User e… view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Performance comparison in ranking using Rank Biased Overlap (RBO) metric for deterministic and probabilistic [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: Selected rank sets for job titles (see Figures 6 in appendix for more examples). [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 6.** Figure 6: Selected rank sets for jobs [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

8 extracted references · 7 canonical work pages

[1]

Marianne Bertrand and Sendhil Mullainathan. 2004. Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimi- nation.American economic review94, 4 (2004), 991–1013

work page 2004
[2]

Ivi Chatzi, Eleni Straitouri, Suhas Thejaswi, and Manuel Rodriguez. 2024. Prediction-powered ranking of large language models.Advances in Neural Infor- mation Processing Systems37 (2024), 113096–113133

work page 2024
[3]

Alessandro Fabris, Nina Baranowska, Matthew J Dennis, David Graus, Philipp Hacker, Jorge Saldivar, Frederik Zuiderveen Borgesius, and Asia J Biega. 2024. Fair- ness and bias in algorithmic hiring: A multidisciplinary survey.ACM Transactions on Intelligent Systems and Technology(2024)

work page 2024
[4]

World Economic Forum. 2021. Human-Centred Artificial Intelligence for Human Resources: A Toolkit for Human Resources Profession- als. https://www3.weforum.org/docs/WEF_Human_Centred_Artificial_ Intelligence_for_Human_Resources_2021.pdf

work page 2021
[5]

Bran Knowles, Jason D’Cruz, John T Richards, and Kush R Varshney. 2023. Humble AI.Commun. ACM66, 9 (2023), 73–79

work page 2023
[6]

Reuters. 2018. Insight - Amazon scraps secret AI recruiting tool that showed bias against women. https://www.reuters.com/article/world/insight- amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women- idUSKCN1MK0AG/

work page 2018
[7]

Why Should I Trust You?

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016. 1135–1144

2016
[8]

2006.Hazards of Hiring

Erik Sink. 2006.Hazards of Hiring. Apress, Berkeley, CA, 97–109. https://doi.org/ 10.1007/978-1-4302-0143-4_10 A Selected examples Table 1 compares the performance of rank sets relative to rankings determined by a single point estimate. Generally the agreement between the two is poor, indicating that the surfaced candidates can be arbitrary. Figure 6 show...

work page doi:10.1007/978-1-4302-0143-4_10 2006

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Marianne Bertrand and Sendhil Mullainathan. 2004. Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimi- nation.American economic review94, 4 (2004), 991–1013

work page 2004

[2] [2]

Ivi Chatzi, Eleni Straitouri, Suhas Thejaswi, and Manuel Rodriguez. 2024. Prediction-powered ranking of large language models.Advances in Neural Infor- mation Processing Systems37 (2024), 113096–113133

work page 2024

[3] [3]

Alessandro Fabris, Nina Baranowska, Matthew J Dennis, David Graus, Philipp Hacker, Jorge Saldivar, Frederik Zuiderveen Borgesius, and Asia J Biega. 2024. Fair- ness and bias in algorithmic hiring: A multidisciplinary survey.ACM Transactions on Intelligent Systems and Technology(2024)

work page 2024

[4] [4]

World Economic Forum. 2021. Human-Centred Artificial Intelligence for Human Resources: A Toolkit for Human Resources Profession- als. https://www3.weforum.org/docs/WEF_Human_Centred_Artificial_ Intelligence_for_Human_Resources_2021.pdf

work page 2021

[5] [5]

Bran Knowles, Jason D’Cruz, John T Richards, and Kush R Varshney. 2023. Humble AI.Commun. ACM66, 9 (2023), 73–79

work page 2023

[6] [6]

Reuters. 2018. Insight - Amazon scraps secret AI recruiting tool that showed bias against women. https://www.reuters.com/article/world/insight- amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women- idUSKCN1MK0AG/

work page 2018

[7] [7]

Why Should I Trust You?

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016. 1135–1144

2016

[8] [8]

2006.Hazards of Hiring

Erik Sink. 2006.Hazards of Hiring. Apress, Berkeley, CA, 97–109. https://doi.org/ 10.1007/978-1-4302-0143-4_10 A Selected examples Table 1 compares the performance of rank sets relative to rankings determined by a single point estimate. Generally the agreement between the two is poor, indicating that the surfaced candidates can be arbitrary. Figure 6 show...

work page doi:10.1007/978-1-4302-0143-4_10 2006