Zara: An LLM-based Candidate Interview Feedback System

Ali Ansari; Aruj Mahajan; Nima Yazdani

REVIEW 4 major objections 5 minor 11 references

Zara: An LLM-based Candidate Interview Feedback System

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A production AI interviewer delivers structured candidate feedback and resolves 75% of follow-up questions automatically, with candidate ratings of 4.37 out of 5.

desk verdict A transparent industry case study whose engineering details are useful, but whose headline metrics—especially the 75% resolution rate—are not backed by evidence. read the letter →

arxiv 2507.02869 v1 pith:Y6AT3ZXR submitted 2025-04-21 cs.HC

Nima Yazdani , Aruj Mahajan , Ali Ansari This is my paper · ORCID

classification cs.HC

keywords candidatefeedbackLLMinterviewsystemretrieval-augmentedgenerationchain-of-thoughtpromptingAI-ledinterviewsexperienceNetPromoterScorerecruitmentautomation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Zara is an AI-driven recruitment support system, built on GPT-4o, that runs candidates through practice interviews, conducts AI-led interviews, then produces structured post-interview feedback and answers follow-up questions automatically. The paper claims this deployment is a working case study showing that large language models can deliver personalized, individually tailored candidate feedback at a scale traditional recruiting cannot reach. In a three-day production window, Zara handled 4,820 unsuccessful interviews, 10.7% of those candidates requested detailed feedback, 75% of candidate inquiries were resolved without human intervention, and candidate ratings averaged a Net Promoter Score of 4.37 out of 5. The stated significance is that the hiring feedback gap—most candidates receive no useful feedback after rejection—is largely a logistics problem that LLMs can now address.

What carries the argument

The load-bearing mechanism is the feedback-generation prompt pipeline rather than a novel model. Zara runs GPT-4o through a chain-of-thought procedure: the model first reviews the interview report and extracts two to three strengths and two to three improvement areas, then performs a reflective pass that rewrites the items as specific, encouraging, past-tense guidance while excluding soft-skill commentary; few-shot exemplars keep the output in a fixed JSON structure. The second mechanism is the RAG query handler: candidate questions are vectorized and matched against a database of embedded FAQ questions by cosine similarity, with a threshold that lets only confident matches reach GPT-4o for answer generation. Together these two components convert an open-ended conversational transcript into repeatable, structured candidate support without a human in the loop.

What would settle it

Take a random sample of Zara's feedback emails, have expert recruiters mark each strength and improvement item as supported, contradicted, or not mentioned in the interview transcript, and compute the rate of unsupported claims; if a moderate fraction of delivered feedback misstates a candidate's performance, the promised improvement in candidate experience does not follow from the reported data.

Watch

Extended reading notes

Core claim

The paper's central claim is that a production LLM system can serve as the candidate-facing interviewer and feedback provider in a real hiring pipeline without sacrificing candidate experience. Zara's feedback pipeline uses chain-of-thought prompting: GPT-4o reads the interview transcript, names two to three strengths and two to three improvements, deliberately withholds soft-skill and communication critiques, and then rewrites the tone to be supportive and 'sugar-coated.' Candidate queries are handled by a retrieval-augmented generation layer in which questions are embedded, matched by cosine similarity against an FAQ question bank, and only high-confidence matches get an automated answer. The paper reports that this design resolves 75% of inquiries without a human, earns an NPS of 4.37 out of 5 from 400 ratings, and raises perceived technical question quality and conversational dynamics above the platform's previous non-conversational GPT-4o system and above human-led interviews on the two measured metrics.

Load-bearing premise

The load-bearing premise is that GPT-4o's feedback is factually accurate, non-hallucinatory, and fair across candidate groups, because the paper measures satisfaction but never checks the feedback against the transcript or audits it for demographic bias.

Editorial extensions

If this is right

Unsuccessful candidates can receive role-specific, actionable guidance within the same platform instead of being ghosted, which is the feedback outcome the paper says candidates report missing.
A support team's load shrinks: with 75% of candidate inquiries answered automatically, recruiters can devote attention to the remaining ambiguous or high-stakes questions.
Perceived interview quality can be competitive with human interviews on structured metrics, making AI-led screening a feasible first-round option for high-volume hiring.
The feedback format is standardized across roles, which makes it possible to compare feedback quality and candidate outcomes across job titles over time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper's reported numbers, the same architecture implies a testable trade-off: raising the RAG similarity threshold would raise precision of automated answers at the cost of a lower auto-resolution rate; the paper does not report this curve.
The 'sugar-coated' and past-tense phrasing rules are an untested intervention; a controlled experiment could measure whether such framing changes candidate learning or reapplication behavior relative to neutral feedback.
Satisfaction ratings alone do not establish feedback accuracy; an accuracy audit that compares each feedback line against the transcript would separate perceived helpfulness from factual correctness.
The three-day window and single-platform setting leave open whether the 75% auto-resolution rate and NPS persist at different candidate volumes or in different languages and job families.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Zara is a readable, honestly framed industry case study: an applied report on using GPT-4o, chain-of-thought, few-shot prompting, and RAG to automate candidate feedback and Q&A in a real hiring platform. The authors explicitly call it an applied use case rather than a new algorithm, and they deserve credit for publishing their actual system and user prompts in full. If you want to see one way to wire these off-the-shelf pieces together for HR, this is a useful reference.

The soft spots are exactly where the reader and stress-test put them. The central claim—‘successfully resolves 75% of candidate inquiries’—is unvalidated. The paper never defines what ‘resolved’ means. In Section 4.2, a RAG pipeline sends an automated response when cosine similarity exceeds a threshold; there is no check that the response actually answers the question, no follow-up rate, no candidate rating of helpfulness. So the 75% figure could simply mean ‘an automated reply was dispatched.’ That is a real problem, and it is load-bearing because the abstract and conclusion lean on it.

The other metrics have similar issues. The NPS of 4.37/5 is reported over 400 ratings without explaining how those ratings were sampled or whether they are representative of the 4,820 interviews. The before/after quality scores (8.38→8.60, 7.77→8.27) are presented as ‘significant improvements’ with no variance, no significance testing, and no inter-rater reliability. And there is no ground-truth validation that the generated feedback is accurate or fair—the system is told to be ‘sugar-coated’ and to avoid soft-skills critique, but nothing checks whether the feedback matches the interview transcript. The authors do not hide these limits; they simply do not address them. For a peer-reviewed paper, that is not enough.

That said, the paper does not pretend to be a rigorous study. As a practitioner report, it has value: the prompt templates, the architecture sketch, and the honest description of a three-day deployment are exactly the kind of detail practitioners want. The citation pattern is fine for an industry paper, with relevant prior work on LLM-based recruiting and ghosting.

Who is this for? Someone building an LLM-based candidate support tool will find the prompts and system design useful. A researcher looking for generalizable findings will be disappointed. If this lands in my inbox as an editor, I would send it to peer review rather than desk reject—there is enough substance to justify referee time—but I would brace the reviewers for a heavy revision focused on the evaluation. The engineering is fine; the metrics need to be rebuilt or honestly downgraded to ‘deployment experience’ with clear caveats.

Serious thinker: yes. The authors know what they built and are candid about its scope. The flaws are in the evaluation, not the reasoning. I would not cite the numbers, but I might cite the prompt design as an example.

Best, [Your name]

Referee Report

4 major / 5 minor

Summary. The paper presents Zara, an LLM-based recruitment support system deployed at micro1, as an applied case study rather than a new algorithmic contribution. Zara uses GPT-4o to generate practice interviews, conduct AI-led interviews, produce structured post-interview feedback via chain-of-thought and few-shot prompting, and answer candidate queries through a RAG pipeline with a cosine-similarity threshold. The reported results, collected over a three-day period, include 4,820 unsuccessful interviews with 10.7% requesting feedback, a claim that Zara automatically resolves 75% of candidate inquiries, an average NPS rating of 4.37 out of 5, and internal quality-score improvements for technical question quality and conversational dynamics. The authors state that the feedback-generation approach is open-sourced.

Significance. If the quantitative claims were properly validated, this case study would provide useful practical evidence that an LLM-based system can deliver personalized interview feedback and query resolution at scale, an area with documented candidate dissatisfaction. The paper's strengths are its explicit architecture description, the concrete prompts for feedback generation, the use of guardrails that exclude soft-skill critique, and the fact that the feedback-generation approach is offered as an open-source artifact. However, the evaluation is entirely internal and unaudited: the central 75% resolution metric rests on an undefined notion of 'resolution,' the satisfaction metric is reported with an unconventional NPS scale, and the claimed 'significant' improvements are presented without any statistical support. The contribution is best understood as a systems description that needs a substantially stronger evaluation before its practical claims can be accepted.

major comments (4)

[Section 5.1] The claim that 'Zara's automated query handling system successfully resolves 75% of candidate inquiries without requiring human intervention' is not supported by the evidence reported. The term 'resolved' is never defined; from the pipeline in Section 4.2, an inquiry likely counts as resolved whenever the cosine-similarity threshold is met and an automated response is dispatched, but no data show that the response answers the candidate's question correctly or usefully. To establish the claim, the paper should report the human-follow-up rate for automated replies, candidate ratings of answer helpfulness, and a manual accuracy audit of a random sample of responses against the FAQ.
[Section 5.1] The statement that technical question quality 'improved significantly' from 8.38 to 8.60 and conversational dynamics from 7.77 to 8.27 is not backed by any statistical evidence. No sample size, standard deviation, rating scale, or significance test is provided, and the comparisons with human-led interviews (7.78 and 5.49) likewise appear without measures of uncertainty. The paper should either report the relevant descriptive statistics with confidence intervals and appropriate tests, or replace 'significantly' with more cautious language such as 'modestly improved in internal ratings.'
[Section 5.1] The Net Promoter Score is conventionally defined on a 0–10 scale and reported as a value between -100 and +100; reporting 'an average NPS rating of 4.37 out of 5' is internally inconsistent and suggests that a different satisfaction scale was used. The paper should clarify the exact question, the numeric scale, the sampling procedure for the 400 ratings, the response rate relative to the candidate population, and the time window over which the ratings were collected.
[Sections 4.1 and 5.2] There is no ground-truth evaluation of the accuracy, non-hallucination, or fairness of the generated feedback. The examples in Figures 7 and 8 look plausible, but the paper does not compare the generated feedback against the actual interview transcripts, nor does it report inter-rater reliability, a hallucination audit, or a fairness check across demographic groups. Since this feedback is delivered to rejected candidates, incorrect or biased statements would directly undermine the stated goal of improving candidate experience; a transcript-based audit with human raters is needed before the claims in Section 5.2 can be accepted.

minor comments (5)

[Figure 5] The caption reads 'Chain-of-Thought prompting technique for Zara candidate query response generation,' but the figure depicts the RAG-based query handling workflow; the caption should be corrected to match the content.
[Section 5.1] The phrase 'There were 4820 “unsuccessful” interviews conducted' would benefit from stating the denominator and time period explicitly, and the 10.7% feedback-request rate should be clarified with respect to which interviews were offered feedback.
[Abstract and Section 4.1] The paper says the feedback-generation approach is open-sourced but no repository URL or artifact identifier is provided; please include a stable link or version reference.
[References] The references 'C E (2020)' and 'O L (2024)' are incomplete; the authors' full names and the publication venues should be supplied.
[Section 5.1] Figures 6a and 6b would be more informative with error bars or confidence intervals; if the underlying data are unavailable, the figure captions should explicitly state that the values are point estimates from internal evaluations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: Zara is an applied case study with usage metrics and system descriptions; the 75% resolution claim is under-validated but not defined circularly.

full rationale

This paper is an applied LLM-system case study, not a derivation or prediction chain. There are no equations, fitted parameters, or first-principles results whose outputs reduce to their inputs. The feedback-generation and query-answering pipelines are described procedurally, and the reported figures are aggregate usage and satisfaction metrics. No self-citations are load-bearing; the references are external prior work. The 75% automatic-resolution figure is the most questionable claim because the paper never defines what counts as 'resolved' and never measures answer correctness or downstream human follow-up, but it also never equates 'resolved' with 'automated response triggered,' so the reduction is not exhibited by construction. The internal evaluations and sample reports are unvalidated credibility concerns, not circularity. Accordingly, no specific circular step can be quoted, and the appropriate honest finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper introduces no mathematical derivation; it rests on domain assumptions about LLM reliability and the meaningfulness of internal metrics. No new physical or conceptual entities are postulated.

free parameters (1)

RAG similarity threshold = unspecified
Zara's RAG workflow matches questions only above an unstated cosine similarity threshold; the threshold controls the 75% automation rate and is not reported in Section 4.2.

assumptions (3)

domain assumption GPT-4o-generated feedback is accurate, non-hallucinatory, and fair.
The paper does not validate feedback quality against ground truth; Section 4.1 only describes prompt instructions.
domain assumption Internal quality ratings and NPS are reliable proxies for candidate experience.
Section 5.1 reports these metrics without describing rating procedures or inter-rater reliability.
domain assumption The 3-day observation window is representative of longer-term system behavior.
Section 5.1 states all usage data comes from a 3-day period.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Zara: An LLM-based Candidate Interview Feedback System." pith.science (2026). https://pith.science/paper/Y6AT3ZXR

@misc{pith2026250702869,
  author       = {Pith},
  title        = {Pith review of: Zara: An LLM-based Candidate Interview Feedback System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y6AT3ZXR}},
  note         = {Machine review of arXiv:2507.02869}
}

read the original abstract

This paper introduces Zara, an AI-driven recruitment support system developed by micro1, as a practical case study illustrating how large language models (LLMs) can enhance the candidate experience through personalized, scalable interview support. Traditionally, recruiters have struggled to deliver individualized candidate feedback due to logistical and legal constraints, resulting in widespread candidate dissatisfaction. Leveraging OpenAI's GPT-4o, Zara addresses these limitations by dynamically generating personalized practice interviews, conducting conversational AI-driven assessments, autonomously delivering structured and actionable feedback, and efficiently answering candidate inquiries using a Retrieval-Augmented Generation (RAG) system. To promote transparency, we have open-sourced the approach Zara uses to generate candidate feedback.

Figures

Figures reproduced from arXiv: 2507.02869 by the authors.

**Figure 2.** An example profile report generated by Zara and added to micro1’s talent pool. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Sample practice interviews generated per role and skill set. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Chain-of-Thought prompting technique for Zara interview feedback generation. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Chain-of-Thought prompting technique for Zara candidate query response generation. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Comparative evaluation of interview quality metrics before and after Zara’s implementation. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Example Feedback Report for a Frontend Developer candidate provided by Zara. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

**Figure 8.** Figure 8: Example Feedback Report for a Product Manager candidate provided by Zara. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 6 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter doi edition editor eid howpublished institution isbn issn journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in "" FUNCTION format.date year ...
[3]

https://www.linkedin.com/pulse/refusing-give-interview-feedback-luxury-employers-elaine-mead/, published August 16, 2020

C E (2020) Refusing to give interview feedback is a luxury employers cannot afford. https://www.linkedin.com/pulse/refusing-give-interview-feedback-luxury-employers-elaine-mead/, published August 16, 2020. LinkedIn article

work page 2020
[4]

arXiv preprint arXiv:2410.20739

Kong H, Ahn Y, Lee S, Maeng Y (2024) Gender bias in llm-generated interview responses. arXiv preprint arXiv:2410.20739

arXiv 2024
[5]

Education and Information Technologies 29(9):11483--11515

Lee U, Jung H, Jeon Y, Sohn Y, Hwang W, Moon J, Kim H (2024) Few-shot is enough: exploring chatgpt prompt engineering method for automatic question generation in english education. Education and Information Technologies 29(9):11483--11515

work page 2024
[6]

Tourism and hospitality management 29(4):583--592

Limna P, Kraiwanit T (2023) The role of chatgpt on customer service in the hospitality industry: An exploratory study of hospitality workers' experiences and perceptions. Tourism and hospitality management 29(4):583--592

work page 2023
[7]

arXiv preprint arXiv:2405.19699

Mujtaba MNR Dena F (2024) Fairness in ai-driven recruitment: Challenges, metrics, methods, and future directions. arXiv preprint arXiv:2405.19699

arXiv 2024
[8]

LinkedIn article

O L (2024) Rejection feedback: Is it really that important? https://www.linkedin.com/pulse/rejection-feedback-really-important-loretta-odoro-acipm-hrpl--taqmf/, published August 5, 2024. LinkedIn article

work page 2024

Show all 11 references

[9]

(2024) Chatgpt and service: opportunities, challenges, and research directions

Sigala M, Ooi KB, Tan GWH, Aw ECX, Cham TH, Dwivedi YK, Kunz WH, Letheren K, Mishra A, Russell-Bennett R, et al. (2024) Chatgpt and service: opportunities, challenges, and research directions. Journal of Service Theory and Practice 34(5):726--737

2024
[10]

(2022) Chain-of-thought prompting elicits reasoning in large language models

Wei J, Wang X, Schuurmans D, Bosma M, Xia F, Chi E, Le QV, Zhou D, et al. (2022) Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35:24824--24837

2022
[11]

Wood NR, Leckfor C, Wicks S, Hales A (2023) Ghosting from the workplace: The impact of feedback (or lack thereof) on applicants’ psychological needs satisfaction

2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter doi edition editor eid howpublished institution isbn issn journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in "" FUNCTION format.date year ...

[3] [3]

https://www.linkedin.com/pulse/refusing-give-interview-feedback-luxury-employers-elaine-mead/, published August 16, 2020

C E (2020) Refusing to give interview feedback is a luxury employers cannot afford. https://www.linkedin.com/pulse/refusing-give-interview-feedback-luxury-employers-elaine-mead/, published August 16, 2020. LinkedIn article

work page 2020

[4] [4]

arXiv preprint arXiv:2410.20739

Kong H, Ahn Y, Lee S, Maeng Y (2024) Gender bias in llm-generated interview responses. arXiv preprint arXiv:2410.20739

arXiv 2024

[5] [5]

Education and Information Technologies 29(9):11483--11515

Lee U, Jung H, Jeon Y, Sohn Y, Hwang W, Moon J, Kim H (2024) Few-shot is enough: exploring chatgpt prompt engineering method for automatic question generation in english education. Education and Information Technologies 29(9):11483--11515

work page 2024

[6] [6]

Tourism and hospitality management 29(4):583--592

Limna P, Kraiwanit T (2023) The role of chatgpt on customer service in the hospitality industry: An exploratory study of hospitality workers' experiences and perceptions. Tourism and hospitality management 29(4):583--592

work page 2023

[7] [7]

arXiv preprint arXiv:2405.19699

Mujtaba MNR Dena F (2024) Fairness in ai-driven recruitment: Challenges, metrics, methods, and future directions. arXiv preprint arXiv:2405.19699

arXiv 2024

[8] [8]

LinkedIn article

O L (2024) Rejection feedback: Is it really that important? https://www.linkedin.com/pulse/rejection-feedback-really-important-loretta-odoro-acipm-hrpl--taqmf/, published August 5, 2024. LinkedIn article

work page 2024