REVIEW 1 cited by
Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Models (LLMs) such as OpenAI Codex are increasingly being used as AI-based coding assistants. Understanding the impact of these tools on developers' code is paramount, especially as recent work showed that LLMs may suggest cybersecurity vulnerabilities. We conduct a security-driven user study (N=58) to assess code written by student programmers when assisted by LLMs. Given the potential severity of low-level bugs as well as their relative frequency in real-world projects, we tasked participants with implementing a singly-linked 'shopping list' structure in C. Our results indicate that the security impact in this setting (low-level C with pointer and array manipulations) is small: AI-assisted users produce critical security bugs at a rate no greater than 10% more than the control, indicating the use of LLMs does not introduce new security risks.
Forward citations
Cited by 1 Pith paper
-
HiLDe: Intentional Code Generation via Human-in-the-Loop Decoding
HiLDe, a code completion UI that exposes and lets users override the LLM's token-level choices, reduced security vulnerabilities in generated code compared to a baseline assistant in a within-subjects study of 18 programmers.
Discussion (0). Sign in to comment.