Pith. sign in

REVIEW

Evaluation and Continual Improvement for an Enterprise AI Assistant

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.12003 v2 pith:WLOLTAU2 submitted 2024-06-15 cs.HC

Akash V. Maharaj , Kun Qian , Uttaran Bhattacharya , Sally Fang , Horia Galatanu , Manas Garg , Rachel Hanessian , Nishant Kapoor
show 3 more authors
Ken Russell Shivakumar Vaithyanathan Yunyao Li
This is my paper · ORCID
classification cs.HC
keywords assistantassistantschallengescontinualdevelopmentevaluationimprovementactive
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

The development of conversational AI assistants is an iterative process with multiple components. As such, the evaluation and continual improvement of these assistants is a complex and multifaceted problem. This paper introduces the challenges in evaluating and improving a generative AI assistant for enterprises, which is under active development, and how we address these challenges. We also share preliminary results and discuss lessons learned.

Discussion (0). Continue with ORCID to comment.

Pith tools