Pith. sign in

REVIEW 4 major objections 5 minor 87 references

If You Had to Pitch Your Ideal Software -- Evaluating Large Language Models to Support User Scenario Writing for User Experience Experts and Laypersons

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM writing assistant lifts laypeople's user scenarios to expert-level quality.

desk verdict A useful, honest empirical study of LLM-assisted user scenario writing, but the headline expert-vs-layperson comparison is confounded by task asymmetry and in-lab rating, so the abstract overreaches. read the letter →

arxiv 2506.23694 v1 pith:UW63C567 submitted 2025-06-30 cs.HC

Patrick Stadler , Christopher Lazik , Christopher Katins , Thomas Kosch This is my paper
classification cs.HC
keywords userscenariosrequirementsanalysislargelanguagemodelswritingassistantzero-shotpromptingfew-shotUXexpertiseusecasegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether people with no UX training can write user scenarios—narrative descriptions of a user's role, routine, challenges, and goals used in requirements analysis—when an LLM-based writing assistant guides them. In a study of 60 participants, the authors report that the assistant condition had a significant effect on scenario quality ($p = 0.0018$), with the zero-shot assistant outperforming no assistant, while expertise alone made no significant difference. Laypersons with the assistant produced scenarios rated comparable to UX experts on structure and clarity, and especially strong on audience-orientation; fourteen of the twenty-five best-rated scenarios were written by laypersons. The paper concludes that LLM-supported assistants can empower laypersons to write good user scenarios and supply first-hand insights that expert-written personas may miss.

What carries the argument

The central object is the writing assistant: a small overlay with two LLM-backed actions, 'Help with the story' and 'Suggest a next step,' plus a completeness estimator and a separate LLM-based use-case preview generator. Zero-shot prompts contain task instructions only; few-shot prompts append ten example scenarios. The assistant's design deliberately returns suggestions under 300 characters and leaves them optional, so the participant retains authorship while being pushed toward the standard scenario anatomy of role, routine, technology, collaboration, challenges, and goals. This scaffold of structured prompts, not the LLM's text generation per se, is what the authors credit for the gains in structure and audience-orientation.

What would settle it

Re-rate the 60 scenarios with independent raters who do not know the condition and do not see the writing-assistant logs; if the zero-shot condition no longer scores above the no-assistant condition, or if the effect disappears once scenario length is controlled for, the claim that the assistant caused the quality gain would fail.

Watch

Extended reading notes

Core claim

The authors claim that an LLM-backed writing assistant, deployed as an optional overlay in the editor, lets UX novices author user scenarios whose rated quality is indistinguishable from those of UX experts, and sometimes superior in audience-orientation. The evidence is a $2 \times 3$ between-subjects study with 60 participants equally split by expertise and assigned to no assistant, a zero-shot assistant, or a few-shot assistant. The zero-shot assistant produced significantly higher ratings than no assistant, the few-shot assistant did not differ significantly from either, and expertise was not a significant factor. Qualitative analysis of the best scenarios shows that laypersons embedded concrete first-hand details about daily routine, tools, and pain points, while weaker scenarios, regardless of group, lacked role introduction and audience awareness. The paper's load-bearing conclusion is that laypersons can be empowered to write good user scenarios, and that these scenarios can be translated into formal use cases by a separate LLM.

Load-bearing premise

The strongest load-bearing premise is that the rating procedure is valid and unbiased: all scenarios were scored by two in-lab researchers using a 24-item checklist, the paper does not state that raters were blind to condition, and expertise groups wrote under different instructions (laypersons about their own jobs, experts about invented third parties), so any of these could distort the measured quality difference.

Editorial extensions

If this is right

  • Laypeople can be brought into requirements analysis as first-hand scenario authors, giving designers concrete daily-routine and pain-point data without requiring UX training.
  • Zero-shot prompt guidance appears sufficient for improving scenario quality; adding few-shot examples did not show a further significant gain in this sample.
  • Because expertise alone showed no significant effect, the traditional assumption that scenario writing requires UX professionals is weakened; the practical bottleneck shifts to designing effective scaffolding.
  • LLM-generated use-case previews, though rated moderately high, tended to produce irrelevant or overly specific use cases from descriptive segments, so scenario-to-use-case translation still needs human review.
  • Writing assistants that only add suggestions risk bloated, over-long scenarios; the observed cycle of completeness-chasing identifies a concrete failure mode to design against.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One implication left implicit is that the zero-shot advantage may reflect prompt-design choice rather than a property of LLMs; few-shot examples drawn from the participants' own job domains might reverse the comparison.
  • Since the conclusion rests on two in-lab raters using one checklist, a natural extension is a blind, independent-rater replication; if the zero-shot effect did not survive blinding, the causal claim would need to be weakened.
  • The authors treat the completeness estimator as a neutral progress indicator, but their data suggest it may function as a motivational nudge; an assistant with the same completeness feedback but no LLM suggestions would isolate that mechanism.
  • The audience-orientation advantage could be operationalized further, for example by asking downstream software engineers to extract requirements from scenarios blindly, connecting the qualitative rating to a direct engineering outcome.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports a between-subjects user study (60 Prolific participants, 30 UX experts and 30 laypersons) in a 2×3 design crossing expertise with writing-assistant condition (none, zero-shot, few-shot). Participants wrote user scenarios in a custom web editor, optionally using an LLM-backed writing assistant; two in-lab researchers rated all scenarios with a 24-item checklist adapted from Michailidou et al., and LLM-generated use-case previews were also rated. A robust two-way ANOVA found a significant main effect of writing-assistant condition (p = 0.0018), with a Tukey post-hoc showing zero-shot versus none differing significantly (p = 0.040), while expertise (p = 0.583) and the interaction (p = 0.372) were not significant. The paper combines these statistics with qualitative analyses of scenario anatomy, writing-assistant usage patterns, and use-case extraction, and concludes that laypersons can be empowered by LLM-supported writing assistants to write user scenarios comparable to those of UX experts in structure and clarity, while particularly excelling in audience-orientation.

Significance. If the central claim held, the paper would make a useful contribution to participatory requirements analysis by showing that non-experts with LLM support can produce usable user scenarios, and the study is comparatively transparent: the prompt designs are given in the appendix, the rating checklist is specified, inter-rater reliability is reported, and the quantitative analysis uses a robust ANOVA justified by a variance test. The qualitative examples of good and poor scenarios and the documented 'chatbot-like' appropriation of the assistant are valuable design insights. However, the main between-group claim is weakened by a task confound: experts wrote about an invented third person while laypersons wrote about their own job, so the null expertise effect and the audience-orientation advantage cannot be attributed to writing skill. The rating procedure also lacks stated blinding, and the exclusion of eight submissions is not fully reported by condition. These issues affect the abstract's central comparison, although the within-layperson effect of the zero-shot assistant is less affected.

major comments (4)
  1. [§4.3 Task] The expertise comparison is confounded with information access. UX experts were instructed to write a user scenario for a third person (e.g., a colleague, partner, or friend), while laypersons described their own profession, workflow, and challenges. The null main effect of expertise (Figure 7, p = 0.583) and the claim that laypersons 'especially excel at audience-orientation' are both explainable by this asymmetry: laypersons had first-hand access to the scenario content, while experts had to invent or infer it. Even a perfectly blinded rating would not separate scenario-writing skill from task-perspective advantage. The Limitations section acknowledges 'Different Initial Positions,' but the abstract and Section 9 still assert comparability of the two groups. The authors should either redesign the comparison so both groups write from the same perspective (e.g., both about their own work, or both about a provided persona) or explicitly reframe the claim as a comparison of two different task types rather than of writing ability.
  2. [§3 and §5.1 Rating procedure] The ratings that drive the central quantitative claim were produced by two researchers from the authors' lab, and the manuscript does not state whether the raters were blind to the experimental condition or to the study hypotheses. Without blinding, the significant writing-assistant effect and the expertise comparison are at risk of expectation bias. The authors should report whether raters were blind, and if not, treat the rating results as preliminary or add independent raters who are blind to condition. Reporting per-rater scores and item-level inter-rater agreement would also help assess whether the 24-item checklist was applied consistently across the five categories.
  3. [§4.1 Participant screening and Section 7 Limitations] Eight submissions were rejected and resampled, and Section 7 states that in some cases submissions were removed 'as a precaution, even without definitive proof of them being AI-generated.' If exclusions were correlated with condition or expertise, the remaining sample is no longer balanced by randomization alone. The authors should report the number of exclusions per condition and per expertise group, the reasons, and ideally a robustness analysis that includes or excludes borderline cases to show the main ANOVA result is not driven by this selective removal.
  4. [Figure 3 and RQ1] The abstract's claim that laypersons 'especially excel at audience-orientation' is not supported by inferential statistics. Figure 3(b) shows descriptive differences in the Audience-Orientation subscale, but no per-category ANOVA or post-hoc test is reported, and the main analysis in Figure 7 collapses all 24 items into a single rating. Given the task confound described above, this claim should either be backed by a per-category statistical test with an appropriate multiple-comparison correction or be demoted to a qualitative observation.
minor comments (5)
  1. [§4.2 and Table 1] The 'no writing assistant' conditions (Baseline and A) still included a minimal assistant with completeness checking, preview, and save functions. The authors should consistently refer to these as 'minimal assistance' rather than 'no WA' in the text and figures, since the significant zero-shot versus none contrast is specifically about LLM-supported suggestions, not the absence of any tool.
  2. [§5.1] Krippendorff's alpha is reported as α = 0.797 without a confidence interval or a statement about which rating level (overall or per item) it was computed on; reporting the interval and the per-category alphas would strengthen the reliability claim.
  3. [§5.2] The effect size η² = 0.104 is reported without a confidence interval, and the post-hoc comparisons in Figure 8 do not include effect sizes; adding these would help readers judge the magnitude of the zero-shot advantage.
  4. [§5.1 Qualitative analysis] The in-depth analysis section is rich, but the coding procedure is described only briefly; stating whether the two raters coded independently and how disagreements were resolved would make the qualitative findings more reproducible.
  5. [References] Some references are incomplete or inconsistently formatted (e.g., [39] and [52] lack venue details, and several arXiv preprint entries lack arXiv identifiers); a final reference cleanup would improve the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

Empirical user study with no circular derivation; the central claims rest on external ratings and collected data, not on self-referential reasoning.

full rationale

This paper is an empirical evaluation, not a mathematical derivation, so the standard circularity patterns do not apply in their usual form. The central claim that LLM-supported writing assistance empowers laypersons to write user scenarios comparable to UX experts is supported by a 60-participant between-subjects study, external rating items from Michailidou et al. [40], an inter-rater reliability statistic (Krippendorff's alpha = 0.797), and statistical tests (robust two-way ANOVA). The rating checklist is an external instrument; the two in-lab raters are a measurement-reliability issue, not a case of the outcome being defined by the input. The paper does not fit parameters to the data and then rename the fit as a prediction, and there is no load-bearing self-citation chain: the cited prior work on user scenarios, use cases, and LLM-based assistance is contextual rather than constitutive of the result. The Limitations section explicitly acknowledges a design asymmetry ('Different Initial Positions') in which laypersons wrote from their own perspective while UX experts wrote for a third person; this is a potential validity confound and a threat to the between-group comparison, but it is not circular reasoning. Likewise, the paper's use of few-shot examples drawn partially from 'our own data collection' influences the few-shot condition, but this is a design choice about prompt construction, not a derivation that reduces the outcome to its inputs. No equation, definition, or citation makes the conclusion equivalent to the study's setup. Therefore, the appropriate circularity finding is a non-finding with minimal concern; the substantive risks are measurement validity and task confounds, which fall outside the circularity construct this pass is designed to identify.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The analysis has no fitted free parameters and introduces no new theoretical entities. It relies on the rating checklist, participant screening, and rater assumptions listed above.

assumptions (3)
  • domain assumption The 24-item checklist from Michailidou et al. [40] is a valid operationalization of user scenario quality.
    Used as the sole rating instrument in Section 5.1; all quality comparisons and the central claim rest on it.
  • domain assumption Prolific screening criteria validly separate UX experts from laypersons.
    Section 4.1 assigns participants to groups based on self-reported job functions and device usage; the expertise comparison depends on this grouping.
  • domain assumption The two in-lab raters produced unbiased scores despite not being stated as blind to condition.
    Section 5.1 reports two raters from the authors' lab; if condition knowledge influenced scores, the writing-assistant effect could be inflated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of If You Had to Pitch Your Ideal Software -- Evaluating Large Language Models to Support User Scenario Writing for User Experience Experts and Laypersons." pith.science (2026). https://pith.science/paper/UW63C567

@misc{pith2026250623694,
  author       = {Pith},
  title        = {Pith review of: If You Had to Pitch Your Ideal Software -- Evaluating Large Language Models to Support User Scenario Writing for User Experience Experts and Laypersons},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UW63C567}},
  note         = {Machine review of arXiv:2506.23694}
}
read the original abstract

The process of requirements analysis requires an understanding of the end users of a system. Thus, expert stakeholders, such as User Experience (UX) designers, usually create various descriptions containing information about the users and their possible needs. In our paper, we investigate to what extent UX novices are able to write such descriptions into user scenarios. We conducted a user study with 60 participants consisting of 30 UX experts and 30 novices who were asked to write a user scenario with or without the help of an LLM-supported writing assistant. Our findings show that LLMs empower laypersons to write reasonable user scenarios and provide first-hand insights for requirements analysis that are comparable to UX experts in terms of structure and clarity, while especially excelling at audience-orientation. We present our qualitative and quantitative findings, including user scenario anatomies, potential influences, and differences in the way participants approached the task.

Figures

Figures reproduced from arXiv: 2506.23694 by the authors.

Figure 1
Figure 1. Screenshot of the web interface used to conduct the main study. Participants were asked to write down their user scenarios ○1 . The LLM-based writing assistant ○2 helps users with two different LLM-supported options ○3 , including improving the current user scenario, and suggesting a next step. Note, that the button in the red rectangle is only presented in the full writing assistant, not the minimal version (baseli… view at source ↗
Figure 2
Figure 2. Close-Up of the LLM-backed writing assistant, as presented to the participants, showing a suggestion that asks to describe the main bottlenecks [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Direct comparison of all user scenario ratings per condition (no writing assistant NO WA, zero-shot writing assistant ZERO SHOT and few-shot writing assistant FEW SHOT), and category, as well as the ratings of the generated use cases for reference. platform where CIF files or other similar chem￾ical file formats can be uploaded and standard￾ized between software platforms, or even mod￾ified depending on which softwa… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Visualization of the user scenario written by UX expert shepherdornate with the zero-shot writing assistant, including all incorporated suggestions. The blue highlighted segments were written after the participant had invoked the writing assistant. It is clearly recogn…
Figure 5
Figure 5. Figure 5: Comparison of writing assistant usage between UX experts and laypeople showing when a participant used the suggestions or ignored or rejected them over time according to the procedure as described at the beginning of this section. All timestamps were normalized. Gaps i…
Figure 8
Figure 8. Figure 8: Results of a post-hoc comparison show a signifi￾cant difference between users with the zero-shot writing as￾sistant (ZERO) and users without any write assistant (none). We found no significant differences between zero-shot and few-shot (FEW) writing assistants, and bet…
Figure 7
Figure 7. Figure 7: Results of a two-way ANOVA for expertise (UX expert, laypeople) and condition (no writing assistant, zero￾shot, and few-shot writing assistant). We found a statistically significant difference between users with and without the writing assistant. layperson) and the wri…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

87 extracted references · 69 canonical work pages

  1. [1]

    Aysh Al-Hroob, Ayad Tareq Imam, and Rawan Al-Heisa. 2018. The use of artificial neural networks for extracting actions and actors from requirements document. Information and Software Technology 101 (2018), 1–15

    2018

  2. [2]

    Bente Anda, Dag Sjøberg, and Magne Jørgensen. 2001. Quality and understandability of use case models. In ECOOP 2001—Object-Oriented Programming: 15th European Conference Budapest, Hungary, June 18– 22, 2001 Proceedings 15 . Springer, 402–428

    2001

  3. [3]

    Deepali Bajaj, Anita Goel, SC Gupta, and Hunar Batra. 2022. MUCE: a multilingual use case model extractor using GPT-3. International Journal of Information Technology 14, 3 (2022), 1543–1554

    2022

  4. [4]

    John M Carrol. 1999. Five Reasons for Scenario-Based Design. In Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. Hicss-32. Abstracts and Cd-Rom of Full Papers . IEEE, 11–pp

    1999

  5. [5]

    Tuhin Chakrabarty, Philippe Laban, and Chien-Sheng Wu. 2024. Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits. arXiv preprint arXiv:2409.14509 (2024)

    arXiv 2024

  6. [6]

    Alistair Cockburn. 1997. Structuring use cases with goals. Journal of object-oriented programming 10, 5 (1997), 56–62

    1997

  7. [7]

    Mike Cohn. 2004. User Stories Applied: For Agile Software Development . Addison-Wesley Professional

    2004

  8. [8]

    Christophe Damas, Bernard Lambeau, Pierre Dupont, and Axel Van Lamsweerde. 2005. Generating annotated behavior models from end-user scenarios. IEEE Transactions on Software Engineering 31, 12 (2005), 1056–1073

    2005

Show all 87 references
  1. [9]

    Gabriele De Vito, Fabio Palomba, Carmine Gravino, Sergio Di Martino, and Filomena Ferrucci. 2023. Echo: An Approach to Enhance Use Case Quality Exploiting Large Language Models. In 2023 49th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) . IEEE, 53–60

    2023

  2. [10]

    Benjamin D Douglas, Patrick J Ewell, and Markus Brauer. 2023. Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA. Plos one 18, 3 (2023), e0279720

    2023

  3. [11]

    Hubert Ekvall and Patrik Winnberg. 2023. Integrating ChatGPT into the UX Design Process: Ideation and Prototyping with Llms

    2023

  4. [12]

    Meryem Elallaoui, Khalid Nafil, and Raja Touahni. 2018. Automatic Transformation of User Stories into UML Use Case Diagrams Using NLP Techniques. Procedia computer science 130 (2018), 42–49

    2018

  5. [13]

    Xiaoxuan Fang, Davy Tsz Kit Ng, Jac Ka Lok Leung, and Samuel Kai Wah Chu. 2023. A Systematic Review of Artificial Intelligence Technologies Used for Story Writing. Education and Information Technologies 28, 11 (2023), 14361–14397

    2023

  6. [14]

    Alessio Ferrari, Sallam Abualhaija, and Chetan Arora. 2024. Model Generation from Requirements with LLMs: An Exploratory Study. arXiv preprint arXiv:2404.06371 (2024). arXiv:2404.06371

    2024 arXiv

  7. [15]

    Alessio Ferrari, Sallam Abualhaijal, and Chetan Arora. 2024. Model Generation with Llms: From Requirements to UML Sequence Dia- grams. In 2024 IEEE 32nd International Requirements Engineering Con- ference Workshops (REW). IEEE, 291–300

    2024

  8. [16]

    The Interaction Design Foundation. 2024. What are User Scenarios? https://www.interaction-design.org/literature/topics/user-scenarios

    2024

  9. [17]

    Giorgio Franceschelli and Mirco Musolesi. 2023. On the Creativity of Large Language Models. arXiv preprint arXiv:2304.00008 (2023). arXiv:2304.00008

    2023 arXiv

  10. [18]

    Rosalva E Gallardo-Valencia, Vivian Olivera, and Susan Elliott Sim

  11. [19]

    Katy Ilonka Gero, Vivian Liu, and Lydia Chilton. 2022. Sparks: Inspi- ration for Science Writing Using Language Models. In Proceedings of the 2022 ACM Designing Interactive Systems Conference . 1002–1019

    2022

  12. [20]

    Tanmai Kumar Ghosh, Atharva Pargaonkar, and Nasir U Eisty. 2024. Exploring Requirements Elicitation from App Store User Reviews Using Large Language Models. arXiv preprint arXiv:2409.15473 (2024). arXiv:2409.15473

    2024 arXiv

  13. [21]

    Fabian Gilson, Matthias Galster, and François Georis. 2020. Generating Use Case Scenarios from User Stories. In Proceedings of the Interna- tional Conference on Software and System Processes . 31–40

    2020

  14. [22]

    Carlos Gómez-Rodríguez and Paul Williams. 2023. A Confederacy of Models: A Comprehensive Evaluation of LLMs on Creative Writing. arXiv preprint arXiv:2310.08433 (2023). arXiv:2310.08433

    2023 arXiv

  15. [23]

    Shailja Gupta and Rajesh Ranjan. 2024. Evaluation of Llms Biases to- wards Elite Universities: A Persona-Based Exploration. arXiv preprint arXiv:2407.12801 (2024). arXiv:2407.12801

    2024 arXiv

  16. [24]

    Guntur Budi Herwanto, Gerald Quirchmayr, and A Min Tjoa. 2024. Leveraging NLP Techniques for Privacy Requirements Engineering in User Stories. IEEE access : practical innovations, open solutions (2024)

    2024

  17. [25]

    Hartwig H Hochmair, Levente Juhász, and Takoda Kemp. 2024. Cor- rectness Comparison of ChatGPT-4, Gemini, Claude-3, and Copilot for Spatial Tasks. Transactions in GIS (2024)

    2024

  18. [26]

    Karen Holtzblatt and Hugh Beyer. 1993. Making customer-centered design work for teams. Commun. ACM 36, 10 (1993), 92–103

    1993

  19. [27]

    Munima Jahan, Mohammad Mahdi Hassan, Reza Golpayegani, Golshid Ranjbaran, Chanchal Roy, Banani Roy, and Kevin Schneider. 2024. Automated Derivation of UML Sequence Diagrams from User Stories: Unleashing the Power of Generative AI vs. a Rule-Based Approach. In Proceedings of th...

    2024

  20. [28]

    Pragyan KC, Rocky Slavin, Sepideh Ghanavati, Travis Breaux, and Mitra Bokaei Hosseini. 2024. An Analysis of Automated Use Case Component Extraction from Scenarios Using ChatGPT. arXiv preprint arXiv:2408.03395 (2024). arXiv:2408.03395

    2024 arXiv

  21. [29]

    Darioush Kevian, Usman Syed, Xingang Guo, Aaron Havens, Geir Dullerud, Peter Seiler, Lianhui Qin, and Bin Hu. 2024. Capabilities of Large Language Models in Control Engineering: A Benchmark Study on Gpt-4, Claude 3 Opus, and Gemini 1.0 Ultra. arXiv preprint arXiv:2404.03647 (2...

    2024 arXiv

  22. [30]

    I’m Not Sure, But

    Sunnie SY Kim, Q Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, and Jennifer Wortman Vaughan. 2024. " I’m Not Sure, But... ": Examin- ing the Impact of Large Language Models’ Uncertainty Expression on User Reliance and Trust. In Proceedings of the 2024 ACM Conference on Fai...

    2024

  23. [31]

    Amy J Ko, Brad A Myers, and Htet Htet Aung. 2004. Six Learning Barriers in End-User Programming Systems. In 2004 IEEE Symposium on Visual Languages-Human Centric Computing . IEEE, 199–206

    2004

  24. [32]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems 35 (2022), 22199–22213. Stadler et al

    2022

  25. [33]

    Kristian Kolthoff, Felix Kretzer, Christian Bartelt, Alexander Maedche, and Simone Paolo Ponzetto. 2024. Interlinking User Stories and GUI Prototyping: A Semi-Automatic LLM-based Approach. arXiv preprint arXiv:2406.08120 (2024). arXiv:2406.08120

    2024 arXiv

  26. [34]

    Klaus Krippendorff. 2011. Computing Krippendorff’s alpha-reliability

    2011

  27. [35]

    Daryl Kulak and Eamonn Guiney. 2012. Use Cases: Requirements in Context. Addison-Wesley

    2012

  28. [36]

    Susan Lilly. 1999. Use case pitfalls: top 10 problems from real projects using use cases. In Proceedings of Technology of Object-Oriented Lan- guages and Systems-TOOLS 30 (Cat. No. PR00278) . IEEE, 174–183

    1999

  29. [37]

    Ze-Lin Liu, Zhinan Zhang, and Yong Chen. 2012. A scenario-based approach for requirements management in engineering design. Con- current Engineering 20, 2 (2012), 99–109

    2012

  30. [38]

    Garm Lucassen, Marcel Robeer, Fabiano Dalpiaz, Jan Martijn EM Van Der Werf, and Sjaak Brinkkemper. 2017. Extracting conceptual models from user stories with Visual Narrator. Requirements Engineering 22 (2017), 339–358

    2017

  31. [39]

    Nicola Marsden and Maren Haag. 2016. Stereotypes and Politics: Reflections on Personas. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems . 4017–4031

    2016

  32. [40]

    Ioanna Michailidou, Charlotte Haid, and Udo Lindemann. 2015. How to Evaluate Use Scenarios and Stories. In Design, User Experience, and Usability: Design Discourse: 4th International Conference, DUXU 2015, Held as Part of HCI International 2015, Los Angeles, CA, USA, August 2–...

    2015

  33. [41]

    Ioanna Michailidou, Constantin von Saucken, and Udo Lindemann

  34. [42]

    Indra Kharisma Raharjana, Daniel Siahaan, and Chastine Fatichah

  35. [43]

    Marcel Robeer, Garm Lucassen, Jan Martijn EM Van Der Werf, Fabi- ano Dalpiaz, and Sjaak Brinkkemper. 2016. Automated Extraction of Conceptual Models from User Stories via NLP. In 2016 IEEE 24th Inter- national Requirements Engineering Conference (RE) . IEEE, 196–205

    2016

  36. [44]

    Krishna Ronanki, Christian Berger, and Jennifer Horkoff. 2023. In- vestigating ChatGPT’s Potential to Assist in Requirements Elicitation Processes. In 2023 49th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE, 354–361

    2023

  37. [45]

    Abdul Malik Sami, Zeeshan Rasheed, Muhammad Waseem, Zheying Zhang, Herda Tomas, and Pekka Abrahamsson. 2024. A Tool for Test Case Scenarios Generation Using Large Language Models. arXiv preprint arXiv:2406.07021 (2024). arXiv:2406.07021

    2024 arXiv

  38. [46]

    Eva-Maria Schön, Dominique Winter, María José Escalona, and Jörg Thomaschewski. 2017. Key Challenges in Agile Requirements Engi- neering. In Agile Processes in Software Engineering and Extreme Pro- gramming: 18th International Conference, XP 2017, Cologne, Germany, May 22-26, ...

    2017

  39. [47]

    Nikhil Singh, Guillermo Bernal, Daria Savchenko, and Elena L Glass- man. 2023. Where to Hide a Stolen Elephant: Leaps in Creative Writ- ing with Multimodal Machine Intelligence. ACM Transactions on Computer-Human Interaction 30, 5 (2023), 1–57

    2023

  40. [48]

    Avik Sinha, Stanley M Sutton, and Amit Paradkar. 2010. Text2Test: Automated Inspection of Natural Language Use Cases. In 2010 Third International Conference on Software Testing, Verification and Validation. IEEE, 155–164

    2010

  41. [49]

    Taylor Sorensen, Joshua Robinson, Christopher Michael Rytting, Alexander Glenn Shaw, Kyle Jeffrey Rogers, Alexia Pauline De- lorey, Mahmoud Khalil, Nancy Fulda, and David Wingate. 2022. An information-theoretic approach to prompt engineering without ground truth labels. arXiv ...

    2022 arXiv

  42. [50]

    Saurabh Tiwari and Atul Gupta. 2015. A Systematic Literature Re- view of Use Case Specifications Research. Information and Software Technology 67 (2015), 128–158

    2015

  43. [51]

    Jordi Tost, Marcel Gohsen, Britta Schulte, Fidel Thomet, Mattis Kuhn, Johannes Kiesel, Benno Stein, and Eva Hornecker. 2024. Futuring Ma- chines: An Interactive Framework for Participative Futuring through Human-AI Collaborative Speculative Fiction Writing. (2024)

    2024

  44. [52]

    Phil Turner and Susan Turner. 2011. Is Stereotyping Inevitable When Designing with Personas? Design studies 32, 1 (2011), 30–44

    2011

  45. [53]

    Mieke van der Bijl-Brouwer and Mascha C van der Voort. 2013. Ex- ploring future use: scenario based design. In Advanced design methods for succesful innovation. Design United, 57–77

    2013

  46. [54]

    Helena Vasconcelos, Matthew Jörke, Madeleine Grunde-McLaughlin, Tobias Gerstenberg, Michael S Bernstein, and Ranjay Krishna. 2023. Explanations can reduce overreliance on ai systems during decision- making. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1–38

    2023

  47. [55]

    Beian Wang, Chong Wang, Peng Liang, Bing Li, and Cheng Zeng. 2024. How LLMs Aid in UML Modeling: An Exploratory Study with Novice Analysts. arXiv preprint arXiv:2404.17739 (2024)

    2024 arXiv

  48. [56]

    Jialiang Wei, Anne-Lise Courbis, Thomas Lambolais, Binbin Xu, Pierre Louis Bernard, Gérard Dray, and Walid Maalej. 2024. Getting Inspiration for Feature Elicitation: App Store-vs. LLM-based Approach. In Proceedings of the 39th IEEE/ACM International Conference on Auto- mated S...

    2024

  49. [57]

    Jing Wei, Sungdong Kim, Hyunhoon Jung, and Young-Ho Kim. 2024. Leveraging large language models to power chatbots for collecting user self-reported data. Proceedings of the ACM on Human-Computer Interaction 8, CSCW1 (2024), 1–35

    2024

  50. [58]

    Matti Welin. 2013. User Stories in Software Development. Cleantech insinöörikoulutuksessa (2013), 108

    2013

  51. [59]

    Joel Wester, Sander De Jong, Henning Pohl, and Niels Van Berkel. 2024. Exploring People’s Perceptions of LLM-generated Advice. Computers in Human Behavior: Artificial Humans (2024), 100072

    2024

  52. [60]

    Jules White, Sam Hays, Quchen Fu, Jesse Spencer-Smith, and Dou- glas C Schmidt. 2024. Chatgpt Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design. In Generative AI for Effective Software Development . Springer, 71–108

    2024

  53. [61]

    Qian Yang, Jina Suh, Nan-Chen Chen, and Gonzalo Ramos. 2018. Grounding Interactive Machine Learning Tool Design in How Non- Experts Actually Build Models. In Proceedings of the 2018 Designing Interactive Systems Conference. 573–584

    2018

  54. [62]

    JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1–21

    2023

  55. [63]

    Prospects

    Zheying Zhang, Maruf Rayhan, Tomas Herda, Manuel Goisauf, and Pekka Abrahamsson. 2024. LLM-based Agents for Automating the Enhancement of User Story Quality: An Early Report. arXiv preprint arXiv:2403.09442 (2024). arXiv:2403.09442 If You Had to Pitch Your Ideal Software A Use...

    2024

  56. [67]

    Be specific daily tasks and processes without being to detailed about every little aspect

  57. [68]

    Include information about tools, software, and technologies used on a high-level perspective

  58. [69]

    Describe collaborations and interactions with team members or other departments

  59. [70]

    Highlight any pain points or challenges in the current workflow

  60. [71]

    Mention any goals or desired improvements in the work process

  61. [72]

    Help with Story

    The user can only reply via buttons in the frontend. They have no option to response with free text. # Tone and Format instructions - Highlight important keywords in a bold face using the respective HTML tags, e.g., <b>daily routine</b>. - Always address the user directly and ...

  62. [73]

    If unclear, let the user provide a basic idea of what they work, what their profession is called, to start

  63. [74]

    Daily routine: What does a typical workday look like?

  64. [75]

    Main responsibilities: What are the primary tasks and projects they work on?

  65. [76]

    Tools and technologies: Which software, hardware, or platforms do they use regularly?

  66. [77]

    Collaboration: Who do they work with, and how do they communicate or share information?

  67. [78]

    Challenges: What are the main difficulties or bottlenecks in their current workflow?

  68. [79]

    Suggest next step

    Goals: What improvements or changes would make their work more efficient or enjoyable? Suggest one of these steps at a time to the user. Do not include any explanation or additional comments that are not part of your task. Encourage the user to provide specific examples and an...

  69. [80]

    Identify the main actors (users or roles) involved in the scenario

    Carefully read and analyze the user scenario. Identify the main actors (users or roles) involved in the scenario

  70. [81]

    Look for actions that involve user interaction with the system or significant system processes

    Break down the scenario into discrete activities or interactions that could form individual use cases. Look for actions that involve user interaction with the system or significant system processes

  71. [82]

    For each identified activity or interaction, create a use case containing the following information: Title: Provide a clear, concise title for the use case Actor: Identify the primary actor (user or role) involved Description: Briefly describe the purpose of the use case Preco...

  72. [83]

    Ensure that each use case is self-contained and focused on a single, specific interaction or process

  73. [84]

    Reggie Field is a freelancer on Upwork

    Review your use cases to ensure they accurately represent the activities described in the original user scenario and that they form a cohesive system when considered together. Remember to focus on creating clear, concise, and interconnected use cases that accurately represent ...

  74. [85]

    How could data analysis and presentation be streamlined?

  75. [86]

    Are there ways to enhance collaboration with your global team?

  76. [87]

    Collaboration Zero-Shot Collaboration Few-Shot Great! You’ve provided a detailed user scenario about Alex, a software engineer

    What new technologies or tools could benefit your research? Please share some specific improvements or changes that would make your work as a scientist more effective or enjoyable. Collaboration Zero-Shot Collaboration Few-Shot Great! You’ve provided a detailed user scenario a...

  77. [2007]

    In 2007 Fifth International Workshop on Comparative Evaluation in Requirements Engineering

    Are use cases beneficial for developers using agile requirements?. In 2007 Fifth International Workshop on Comparative Evaluation in Requirements Engineering. IEEE, 11–22

    2007

  78. [2013]

    InDesign, User Experience, and Usability

    How to create a user experience story. InDesign, User Experience, and Usability. Design Philosophy, Methods, and Tools: Second Interna- tional Conference, DUXU 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA, July 21-26, 2013, Proceedings, Part I 2 . Springer, 554–563

    2013

  79. [2021]

    IEEE access : practical innovations, open solutions 9 (2021), 53811–53826

    User Stories and Natural Language Processing: A Systematic Literature Review. IEEE access : practical innovations, open solutions 9 (2021), 53811–53826

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.