Pith. sign in

REVIEW 3 cited by

What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use

Not yet reviewed by Pith; the record is open.

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.08775 v3 pith:DV6T6BMN submitted 2024-09-13 cs.HC cs.AI

classification cs.HCcs.AI
keywords prompttrainingengineeringrequirementsropecomplexhumansprompting
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Prompting LLMs for complex tasks (e.g., building a trip advisor chatbot) needs humans to clearly articulate customized requirements (e.g., "start the response with a tl;dr"). However, existing prompt engineering instructions often lack focused training on requirement articulation and instead tend to emphasize increasingly automatable strategies (e.g., tricks like adding role-plays and "think step-by-step"). To address the gap, we introduce Requirement-Oriented Prompt Engineering (ROPE), a paradigm that focuses human attention on generating clear, complete requirements during prompting. We implement ROPE through an assessment and training suite that provides deliberate practice with LLM-generated feedback. In a randomized controlled experiment with 30 novices, ROPE significantly outperforms conventional prompt engineering training (20% vs. 1% gains), a gap that automatic prompt optimization cannot close. Furthermore, we demonstrate a direct correlation between the quality of input requirements and LLM outputs. Our work paves the way to empower more end-users to build complex LLM applications.

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 5 citations worldwide. Full citation record

  1. ViseGPT: Towards Better Alignment of LLM-generated Data Wrangling Scripts and User Prompts

    cs.HC 2025-08 conditional novelty 6.0 of 10

    ViseGPT automatically converts user prompts into test cases and visualizes which steps of an LLM-generated data wrangling script pass or fail.

  2. Designing Conversational AI to Support Think-Aloud Practice in Technical Interview Preparation for CS Students

    cs.HC 2025-07 conditional novelty 6.0 of 10

    A 17-participant study finds CS students value LLM-based mock interviews for think-aloud practice and want AI that feels more present, gives feedback beyond spoken words, and mixes human examples with AI generation.

  3. From Legal Text to Tech Specs: Generative AI's Interpretation of Consent in Privacy Law

    cs.SE 2025-07 conditional novelty 4.0 of 10

    An LLM pipeline that flags and fixes non-compliant consent use cases works imperfectly: it catches about two-thirds of relevant cases with reasoning prompts, and most of its fixes are legally sound but logically inconsistent.

Pith tools