Pith. sign in

REVIEW

Validating AI-Generated Code with Live Programming

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.09541 v3 pith:NSN27FFQ submitted 2023-06-15 cs.HC cs.PL

Kasra Ferdowsi , Ruanqianqian Huang , Michael B. James , Nadia Polikarpova , Sorin Lerner This is my paper · ORCID
classification cs.HCcs.PL
keywords programmingai-generatedai-poweredchallengecodedevelopersenvironmentlive
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

AI-powered programming assistants are increasingly gaining popularity, with GitHub Copilot alone used by over a million developers worldwide. These tools are far from perfect, however, producing code suggestions that may be incorrect in subtle ways. As a result, developers face a new challenge: validating AI's suggestions. This paper explores whether Live Programming (LP), a continuous display of a program's runtime values, can help address this challenge. To answer this question, we built a Python editor that combines an AI-powered programming assistant with an existing LP environment. Using this environment in a between-subjects study (N=17), we found that by lowering the cost of validation by execution, LP can mitigate over- and under-reliance on AI-generated programs and reduce the cognitive load of validation for certain types of tasks.

Discussion (0). Continue with ORCID to comment.

Pith tools