Pith. sign in

REVIEW 1 cited by

How Far Are We? The Triumphs and Trials of Generative AI in Learning Software Engineering

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.11719 v1 pith:FAU2LSXS submitted 2023-12-18 cs.SE cs.HC

Rudrajit Choudhuri , Dylan Liu , Igor Steinmacher , Marco Gerosa , Anita Sarma This is my paper · ORCID
classification cs.SEcs.HC
keywords chatgptconvo-genaiengineeringgenerativeparticipantssoftwarestudentstasks
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Conversational Generative AI (convo-genAI) is revolutionizing Software Engineering (SE) as engineers and academics embrace this technology in their work. However, there is a gap in understanding the current potential and pitfalls of this technology, specifically in supporting students in SE tasks. In this work, we evaluate through a between-subjects study (N=22) the effectiveness of ChatGPT, a convo-genAI platform, in assisting students in SE tasks. Our study did not find statistical differences in participants' productivity or self-efficacy when using ChatGPT as compared to traditional resources, but we found significantly increased frustration levels. Our study also revealed 5 distinct faults arising from violations of Human-AI interaction guidelines, which led to 7 different (negative) consequences on participants.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Educators' Perceptions of Large Language Models as Tutors: Comparing Human and AI Tutors in a Blind Text-only Setting

    cs.ET 2025-06 conditional novelty 6.0 of 10

    In blind pairwise comparisons, educators rated an LLM tutor (MWPTutor) as better than human tutors from MathDial on empathy, scaffolding, and conciseness, with no significant advantage on engagement.

Pith tools