Pith. sign in

REVIEW 2 cited by

Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Products

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.12071 v2 pith:YRCMOX3T submitted 2024-10-15 cs.SE cs.LG

Nadia Nahar , Christian Kästner , Jenna Butler , Chris Parnin , Thomas Zimmermann , Christian Bird This is my paper · ORCID
classification cs.SEcs.LG
keywords softwarechallengesdevelopersemergingllmsproductssolutionsacross
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large Language Models (LLMs) are increasingly embedded into software products across diverse industries, enhancing user experiences, but at the same time introducing numerous challenges for developers. Unique characteristics of LLMs force developers, who are accustomed to traditional software development and evaluation, out of their comfort zones as the LLM components shatter standard assumptions about software systems. This study explores the emerging solutions that software developers are adopting to navigate the encountered challenges. Leveraging a mixed-method research, including 26 interviews and a survey with 332 responses, the study identifies 19 emerging solutions regarding quality assurance that practitioners across several product teams at Microsoft are exploring. The findings provide valuable insights that can guide the development and evaluation of LLM-based products more broadly in the face of these challenges.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Understanding Prompt Programming Tasks and Questions

    cs.SE 2025-07 conditional novelty 7.0 of 10

    Prompt programmers ask 51 questions across 25 tasks, and existing tools leave 16 of those questions, including the most important ones, unanswered.

  2. A Large-Scale Evolvable Dataset for Model Context Protocol Ecosystem and Security Analysis

    cs.CR 2025-06 conditional novelty 6.0 of 10

    The paper releases MCPCorpus, a large-scale annotated dataset of MCP servers and clients with over 20 normalized attributes, plus tooling for updates and exploration.

Pith tools