REVIEW 2 cited by
Are LLMs Correctly Integrated into Software Systems?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) provide effective solutions in various application scenarios, with the support of retrieval-augmented generation (RAG). However, developers face challenges in integrating LLM and RAG into software systems, due to lacking interface specifications, various requirements from software context, and complicated system management. In this paper, we have conducted a comprehensive study of 100 open-source applications that incorporate LLMs with RAG support, and identified 18 defect patterns. Our study reveals that 77% of these applications contain more than three types of integration defects that degrade software functionality, efficiency, and security. Guided by our study, we propose systematic guidelines for resolving these defects in software life cycle. We also construct an open-source defect library Hydrangea.
Forward citations
Cited by 2 Pith papers
-
Understanding the Supply Chain and Risks of Large Language Model Applications
A new benchmark dataset traces dependencies across 3,859 LLM applications, 109,211 models, 2,474 datasets, and 8,862 libraries, and finds widespread known vulnerabilities in application dependencies.
-
Specification and Detection of LLM Code Smells
A catalog of five LLM code smells and a static detection tool find that 60.5% of 200 open-source LLM-using Python projects exhibit at least one smell.
Discussion (0). Sign in to comment.