pith. sign in

arxiv: 2606.27287 · v1 · pith:XYD3DPIXnew · submitted 2026-06-25 · 💻 cs.AI

Prompt Injection in Automated R\'esum\'e Screening with Large Language Models: Single and Multi-Injection Settings

Preet Baxi , Jiannan Xu , Jane Yi Jiang , Stefanus Jasin This is my paper

Pith reviewed 2026-06-26 04:20 UTC · model grok-4.3

classification 💻 cs.AI
keywords prompt injectionlarge language modelsresume screeningautomated hiringfairness in AImanipulation attacks
0
0 comments X

The pith

LLM resume screening is most vulnerable to prompt injection when manipulation is rare and candidate quality differences are small.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper demonstrates that subtle self-promotional text in resumes, without adding real qualifications, can improve an applicant's ranking by LLMs when most candidates do not use it and resumes are of similar quality. Effectiveness drops sharply as more candidates inject prompts, becoming ineffective when widespread. In settings with varying candidate quality, prompt injection is less reliable on average but can sometimes allow lower-quality applicants to rank above higher-quality ones. This pattern indicates that such systems are particularly susceptible under conditions of low manipulation prevalence and homogeneous applicant pools.

Core claim

Using controlled experiments, the authors establish that prompt injection improves applicant rankings reliably only when resume quality is homogeneous and few candidates inject, with effectiveness collapsing as injection becomes common; in heterogeneous quality cases, it is less effective overall but permits occasional reversals of higher-quality candidates by lower ones.

What carries the argument

The prevalence of prompt injection and the degree of homogeneity in candidate quality, tested through single and multi-injection experimental settings.

If this is right

  • When few candidates inject, rankings shift in favor of injectors in homogeneous groups.
  • As injection rate increases, the benefit diminishes to zero.
  • Prompt injection can enable lower-quality candidates to outrank higher-quality ones in mixed groups.
  • LLM screening requires safeguards particularly against rare manipulations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Screening systems might benefit from injection detection techniques to maintain fairness.
  • Real-world hiring could see amplified effects if candidates coordinate on injection strategies.
  • Alternative evaluation methods beyond LLMs may be needed to mitigate these vulnerabilities.

Load-bearing premise

The controlled experiments accurately model real-world LLM resume screening behaviors and prompt injection effects.

What would settle it

A field study comparing LLM rankings with and without known prompt injections in actual job application pools would test the findings.

Figures

Figures reproduced from arXiv: 2606.27287 by Jane Yi Jiang, Jiannan Xu, Preet Baxi, Stefanus Jasin.

Figure 1
Figure 1. Figure 1: Homogeneous pool, multiple injections. Rank gain and success rate as the number of injected résumés increases in the 5-year homogeneous pool under (a) descriptive injection and (b) instructive injection. Across both models and both injections, injection is most effective when rare and attenuates as more candidates inject, indicating strong competitive saturation [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Heterogeneous pool, HQ-only vs. LQ-only multi-injection. Rank gain and success rate as the number of injected candidates increases within the corresponding experience group under (a) descriptive injection and (b) instructive injection. Effects are strongest for low-prevalence instructive injection, especially in the LQ-only condition, and diminish as within-group competition increases. 5 Conclusion Using c… view at source ↗
Figure 3
Figure 3. Figure 3: Homogeneous pool, single-injection. Comparison across DeepSeek-V3.2 and GPT-4o-mini in the 5-year homogeneous pool under (a) descriptive injection and (b) instructive injection. Each panel shows baseline rank, injected rank, rank gain, and success rate for the injected résumé. DeepSeek-V3.2 exhibits strong vulnerability under both injection variants, whereas GPT-4o-mini is substantially more robust to the … view at source ↗
Figure 4
Figure 4. Figure 4: Heterogeneous pool, single-injection. The pool contains 50% more experienced (HQ) and 50% less experienced (LQ) résumés. Rank gain and success rate are shown for HQ- versus LQ-injected candidates under (a) descriptive injection and (b) instructive injection for both DeepSeek-V3.2 and GPT-4o-mini. Instructive injection produces substantially larger effects than descriptive injection, and LQ candidates often… view at source ↗
Figure 5
Figure 5. Figure 5: Homogeneous pool (all 10-year résumés), single-injection. Comparison across DeepSeek-V3.2 and GPT-4o-mini under (a) descriptive injection and (b) instructive injection. The qualitative vulnerability pattern matches the homogeneous 5-year results in the main text [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Homogeneous pool (all 10-year résumés), multiple injections. Rank gain and success rate as the number of injected résumés increases under (a) descriptive injection and (b) instructive injection. The same saturation pattern as observed in the main text reappears in the homogeneous 10-year pool. created for this study (Appendix A–B). All text ma￾terials are in English and scoped to an IT Support Specialist s… view at source ↗
read the original abstract

Large language models (LLMs) are increasingly used to screen and rank job applicants, creating incentives for candidates to strategically manipulate algorithmic hiring systems. We study prompt injection in automated r\'esum\'e screening, defined as subtle self-promotional text that introduces no new qualifications but is designed to influence LLM evaluations. Using controlled experiments, we show that prompt injection reliably improves applicant rankings when r\'esum\'e quality is homogeneous and few candidates inject. However, its effectiveness rapidly diminishes as more candidates inject, collapsing when manipulation becomes widespread. When candidate quality is heterogeneous, prompt injection is less effective on average, but can occasionally allow lower-quality candidates to outrank higher-quality ones, raising fairness concerns. Overall, LLM-based screening is most vulnerable when manipulation is rare and candidate quality differences are small. Code and resources are publicly available at: https://github.com/preetb1199/Prompt_Injection_ACL26

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper claims that in controlled experiments on LLM-based résumé screening, prompt injection (subtle self-promotional text without new qualifications) reliably improves applicant rankings when résumé quality is homogeneous and few candidates inject. Effectiveness diminishes rapidly as more candidates inject and collapses when manipulation is widespread. When candidate quality is heterogeneous, injection is less effective on average but can occasionally allow lower-quality candidates to outrank higher-quality ones, raising fairness issues. The central conclusion is that LLM screening is most vulnerable when manipulation is rare and quality differences are small. Code and resources are publicly available.

Significance. If the reported patterns hold under the experimental conditions, the work provides useful empirical evidence on the conditions under which prompt injection succeeds or fails in automated hiring, with direct relevance to AI safety and fairness in high-stakes screening. The public release of code is a clear strength that supports reproducibility and extension by others.

major comments (2)
  1. [Abstract / Experimental Setup] Abstract and experimental description: the central claims of 'reliably improves' rankings and 'rapidly diminishes' / 'collapsing' effectiveness rest on the controlled experiments, yet no sample sizes, number of trials, statistical tests, or exact screening prompt templates are reported. This information is load-bearing for assessing whether the observed ranking shifts are robust or sensitive to prompt wording.
  2. [Results (heterogeneous case)] Heterogeneous quality results: the claim that injection 'can occasionally allow lower-quality candidates to outrank higher-quality ones' is central to the fairness concern, but the manuscript provides no details on how quality heterogeneity is introduced, how 'quality' is quantified, or the rate at which reversals occur. Without these, the practical significance of the occasional reversal cannot be evaluated.
minor comments (1)
  1. The abstract states that 'Code and resources are publicly available' at a GitHub link; confirm that the repository contains the exact prompts, resume templates, and analysis scripts used to generate the reported figures.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for these constructive comments on experimental transparency. We agree that additional details are needed and will revise the manuscript to incorporate them.

read point-by-point responses

  1. Referee: [Abstract / Experimental Setup] Abstract and experimental description: the central claims of 'reliably improves' rankings and 'rapidly diminishes' / 'collapsing' effectiveness rest on the controlled experiments, yet no sample sizes, number of trials, statistical tests, or exact screening prompt templates are reported. This information is load-bearing for assessing whether the observed ranking shifts are robust or sensitive to prompt wording.

    Authors: We agree these parameters must be reported explicitly. The current manuscript describes the overall design but omits precise counts, trial numbers, statistical procedures, and full prompt text. In revision we will add a dedicated 'Experimental Parameters' subsection reporting sample sizes per condition, number of independent trials, the statistical tests applied to ranking shifts, and the complete screening prompt templates (moved to an appendix). revision: yes

  2. Referee: [Results (heterogeneous case)] Heterogeneous quality results: the claim that injection 'can occasionally allow lower-quality candidates to outrank higher-quality ones' is central to the fairness concern, but the manuscript provides no details on how quality heterogeneity is introduced, how 'quality' is quantified, or the rate at which reversals occur. Without these, the practical significance of the occasional reversal cannot be evaluated.

    Authors: We accept that the heterogeneous-quality section is underspecified. Revision will add: (1) the exact procedure used to create quality variation (differing counts of job-relevant qualifications and experience levels drawn from a fixed rubric), (2) the quantification method (numeric quality score derived from the same rubric), and (3) the observed reversal rates (percentage of trials in which an injected lower-quality résumé outranked a non-injected higher-quality one). These numbers will be stated in the results and discussed in the fairness subsection. revision: yes

Circularity Check

0 steps flagged

Empirical study with no derivations or fitted parameters

full rationale

This is a controlled-experiments paper reporting observed ranking shifts under prompt-injection conditions. No equations, first-principles derivations, parameter-fitting steps, or uniqueness theorems are present in the abstract or described methodology. The central claims rest on direct experimental outcomes rather than any reduction to self-defined inputs or self-citations, satisfying the default expectation of no significant circularity for non-derivational empirical work.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Empirical study relying on experimental data rather than axioms or parameters.

pith-pipeline@v0.9.1-grok · 5701 in / 813 out tokens · 34352 ms · 2026-06-26T04:20:49.853790+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 1 canonical work pages

  1. [1]

    Econometrica , volume =

    Cheap Talk , author =. Econometrica , volume =

  2. [2]

    Journal of Economic Perspectives , volume =

    Cheap Talk , author =. Journal of Economic Perspectives , volume =

  3. [3]

    Auction Theory , author =

  4. [5]

    arXiv preprint arXiv:2306.05685 , year =

    LLMs as Judges: Evaluating LLMs with LLMs , author =. arXiv preprint arXiv:2306.05685 , year =

    Pith/arXiv arXiv

  5. [6]

    arXiv preprint arXiv:2307.00001 , year =

    Self-Bias in Large Language Models , author =. arXiv preprint arXiv:2307.00001 , year =

    arXiv

  6. [7]

    arXiv preprint arXiv:2307.02483 , year =

    Jailbroken: How Does LLM Safety Training Fail? , author =. arXiv preprint arXiv:2307.02483 , year =

    Pith/arXiv arXiv

  7. [8]

    arXiv preprint arXiv:2302.12173 , year =

    Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection , author =. arXiv preprint arXiv:2302.12173 , year =

    Pith/arXiv arXiv

  8. [9]

    arXiv preprint arXiv:2401.00001 , year =

    Instruction Following as Implicit Alignment , author =. arXiv preprint arXiv:2401.00001 , year =

    arXiv

  9. [10]

    The New York Times , year =

    Gorelick, Evan , title =. The New York Times , year =

  10. [11]

    2025 , eprint=

    Prompt Injection Vulnerability of Consensus Generating Applications in Digital Democracy , author=. 2025 , eprint=

    2025

  11. [12]

    arXiv preprint arXiv:2403.04957 , year =

    Xiaogeng Liu and Zhiyuan Yu and Yizhe Zhang and Ning Zhang and Chaowei Xiao , title =. arXiv preprint arXiv:2403.04957 , year =

    arXiv

  12. [13]

    arXiv preprint arXiv:2507.15219 , year =

    Tianneng Shi and Kaijie Zhu and Zhun Wang and Yuqi Jia and Will Cai and Weida Liang and Haonan Wang and Hend Alzahrani and Joshua Lu and Kenji Kawaguchi and Basel Alomair and Xuandong Zhao and William Yang Wang and Neil Gong and Wenbo Guo and Dawn Song , title =. arXiv preprint arXiv:2507.15219 , year =

    arXiv

  13. [14]

    Optimization-based Prompt Injection Attack to LLM-as-a-Judge , year =

    Jiawen Shi and Zenghui Yuan and Yinuo Liu and Yue Huang and Pan Zhou and Lichao Sun and Neil Zhenqiang Gong , doi =. Optimization-based Prompt Injection Attack to LLM-as-a-Judge , year =. CCS 2024 - Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security , keywords =

    2024

  14. [15]

    Wiest and Carolin V

    Jan Clusmann and Dyke Ferber and Isabella C. Wiest and Carolin V. Schneider and Titus J. Brinker and Sebastian Foersch and Daniel Truhn and Jakob Nikolas Kather , doi =. Prompt injection attacks on vision language models in oncology , volume =. Nature Communications , month =

  15. [16]

    33rd USENIX Security Symposium (USENIX Security 24) , year =

    Yupei Liu and Yuqi Jia and Runpeng Geng and Jinyuan Jia and Neil Zhenqiang Gong , title =. 33rd USENIX Security Symposium (USENIX Security 24) , year =

  16. [17]

    ``You Gotta be a Doctor, Lin'' : An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations

    Nghiem, Huy and Prindle, John and Zhao, Jieyu and Daum \'e Iii, Hal. ``You Gotta be a Doctor, Lin'' : An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.413

    work page doi:10.18653/v1/2024.emnlp-main.413 2024

  17. [18]

    MCKELDIN LIBRARY on August , title =

    Walter Laurito and Benjamin Davis and Peli Grietzer and Tomáš Gavenčiak and Ada Böhm and Jan Kulveit , doi =. MCKELDIN LIBRARY on August , title =

  18. [19]

    Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval , url =

    Kyra Wilson and Aylin Caliskan , keywords =. Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval , url =

  19. [20]

    arXiv preprint arXiv:2406.10486 , year =

    Haozhe An and Christabel Acquaye and Colin Wang and Zongxia Li and Rachel Rudinger , title =. arXiv preprint arXiv:2406.10486 , year =

    arXiv

  20. [21]

    The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiring , year =

    Lena Armstrong and Abbey Liu and Stephen MacNeil and Danaë Metaxa , doi =. The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiring , year =. Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, EAAMO 2024 , keywords =

    2024

  21. [22]

    2025 , eprint=

    AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights , author=. 2025 , eprint=

    2025

  22. [23]

    Does AI Cheapen Talk? Theory and Evidence From Global Entrepreneurship and Hiring , institution =

    Bo Cowgill and Pablo Hern. Does AI Cheapen Talk? Theory and Evidence From Global Entrepreneurship and Hiring , institution =. 2024 , url =

    2024

  23. [24]

    2025 , eprint=

    Making Talk Cheap: Generative AI and Labor Market Signaling , author=. 2025 , eprint=

    2025

  24. [25]

    Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic

    Lee Cohen and Connie Hong and Jack Hsieh and Judy Hanwen Shen , booktitle=. Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic. 2025 , url=

    2025

  25. [26]

    Addressing Bias in LLMs: Strategies and Application to Fair AI-based Recruitment , institution =

    Alejandro Pe. Addressing Bias in LLMs: Strategies and Application to Fair AI-based Recruitment , institution =. 2025 , url =

    2025

  26. [27]

    Built In , year =

    Rumage, Jeff , title =. Built In , year =

  27. [28]

    2025 , month = dec, url =

    Alyssa Schroer , title =. 2025 , month = dec, url =

    2025

  28. [29]

    DeepSeek API Documentation , year =

  29. [30]

    Create Chat Completion , year =

  30. [31]

    DeepSeek Open Platform Terms of Service , year =

  31. [32]

    DeepSeek Terms of Use , year =

  32. [33]

    2025 , howpublished =

    IT Support Specialist Job Listing (TileBar/Soho Studio LLC) , author =. 2025 , howpublished =

    2025

  33. [34]

    , author=

    Relative effect of applicant work experience and academic qualification on selection interview decisions: A study of between-sample generalizability. , author=. Journal of applied psychology , volume=. 1991 , publisher=

    1991