Pith. sign in

REVIEW 4 major objections 4 minor 50 references

To Google or To ChatGPT? A Comparison of CS2 Students' Information Gathering Approaches and Outcomes

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that CS2 students learned a harder programming concept less effectively with ChatGPT than with web search, with a large quiz-score gap on currying.

desk verdict A useful exploratory comparison of ChatGPT vs. web search for CS2 learning, with solid behavioral findings but a headline quiz result that is less secure than the abstract suggests. read the letter →

arxiv 2501.11935 v3 pith:2POGTRG3 submitted 2025-01-21 cs.HC cs.AI

Aayush Kumar , Daniel Prol , Amin Alipour , Sruti Srinivasa Ragavan This is my paper · ORCID
classification cs.HCcs.AI
keywords ChatGPTLLMineducationCS2curryingIIFEinformationseekingself-directedlearningconceptualunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that when CS2 students teach themselves a new programming concept, their information-gathering tool changes both how they search and how well they learn. In a within-subject lab study with 32 students, the harder of two JavaScript concepts, currying, was learned significantly less effectively with ChatGPT than with traditional web search: the average conceptual quiz score was $M=-0.69$ in the AI condition versus $M=2.0$ in the NoAI condition ($p=.005$). For the easier concept, immediately invoked function expressions, quiz scores did not differ significantly between the two conditions. The paper also claims that students adapt their questioning style to the tool, writing more follow-up and question-style prompts to ChatGPT and more keyword-based queries to search engines. If the result holds, the direct-answer interactivity of an LLM can be a disadvantage for initial learning exactly when the topic is difficult.

What carries the argument

The argument is carried by a within-subject lab protocol in which each of 32 CS2 students learned two JavaScript concepts, currying and immediately invoked function expressions, in 15-minute sessions, one with ChatGPT (GPT-3.5 Turbo) and one with web search and video, with task-treatment order balanced. Learning outcomes were measured with a five-question conceptual quiz scored from $-8$ to $+8$ with a "don't know" option, plus a 10-minute debugging task. Information-seeking behavior was coded with a query and prompt taxonomy (base versus follow-up, copied versus edited, and phrasing categories such as Keyword-Based, Explanation, and Evidence-Based) and an activity codebook tracking time spent locating sources, learning by example, learning theory, and using the IDE. The mechanism that carries the claim is the contrast these instruments reveal: a large quiz gap on the harder concept combined with systematic differences in how students phrased requests to the two tools.

What would settle it

A larger preregistered study that randomly assigns students to ChatGPT or web search for currying, records prior JavaScript and programming experience, and finds no quiz-score gap (or a gap favoring ChatGPT) would overturn the central claim.

Watch

Extended reading notes

Core claim

The central claim is that information gathering for self-directed programming learning has different effectiveness depending on topic difficulty, and that traditional web search can beat ChatGPT on a harder concept. On currying, participants using ChatGPT scored an average of $-0.69$ (SD=2.2) on the conceptual quiz, versus $2.0$ (SD=2.06) for participants using Google and YouTube, a statistically significant difference ($p=.005$, effect size $r=0.50$); on the easier IIFE task, the corresponding comparison was not significant. The authors interpret the currying gap as evidence that the multi-source, keyword-driven process of web search yields a more complete understanding of a hard concept, while ChatGPT's direct, on-demand answers may narrow what a learner explores. They also report that participants wrote 237 prompts in the AI condition against 148 queries in the NoAI condition, and that LLM prompts were significantly more likely to be follow-ups and explanation-seeking questions. A debugging task showed no significant treatment difference, which the paper reads as a productivity-versus-learning distinction: LLM assistance can help produce working code without guaranteeing conceptual grasp.

Load-bearing premise

The load-bearing premise is that the 16 students who learned currying with ChatGPT and the 16 who learned it through web search were comparable in programming ability, motivation, and prior exposure, since the paper balances task order but does not measure these traits.

Editorial extensions

If this is right

  • If the currying result generalizes, educators should not treat LLMs as drop-in replacements for web search in self-directed learning of difficult topics.
  • Students using LLMs for learning may need prompting guidance that encourages broad concept coverage rather than narrow follow-up questions.
  • The absence of a quiz difference on the easier concept indicates that blanket claims about LLM efficacy in programming education need to be qualified by topic difficulty.
  • The debugging-task result suggests that code that runs is a weak signal of understanding when AI assistance is available; assessments should probe conceptual knowledge separately.
  • For tool designers, the data imply that making LLM responses more holistic, for instance by pointing to multiple perspectives or external resources, could narrow the learning gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of this result is that the quiz gap between ChatGPT and web search should increase with topic difficulty and persist when learning is tested after a delay rather than immediately.
  • The query-analysis data suggest a mechanism the paper does not directly test: keyword search exposes learners to several curated sources, whereas follow-up prompts on an LLM can confine the learner to one conversational thread; a study that forces the same prompting style in both tools could isolate this cause.
  • The debugging results imply that in classrooms where AI tools are allowed, instructors should evaluate transfer with new problems rather than accepting successful code patches as evidence of learning.
  • A design implication not proven here is that enriching LLM responses with heterogeneous content or multiple suggested perspectives might recover some of web search's breadth; the paper's data only indirectly motivate this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper reports a within-subjects lab study with 32 CS2 students at IIT Kanpur, in which each participant learned two JavaScript concepts—currying and IIFE—one with ChatGPT and one with traditional web-based resources (Google/YouTube). The authors analyze screen recordings, prompts/queries, quiz scores, and debugging performance to compare information-gathering strategies and learning outcomes across the two conditions. The headline finding is that participants scored significantly lower on a conceptual quiz for the more difficult concept (currying) after learning with ChatGPT (M = -0.69) than with web search (M = 2.0, p = .005), which the authors interpret as evidence that LLMs may be less effective for learning difficult concepts. Secondary findings include that LLM prompts are more likely to be follow-ups and that search-engine queries are more keyword-based.

Significance. If the headline result holds, the paper makes a meaningful contribution to the computing-education literature by providing a controlled comparison of LLM-based and web-based self-learning, with a mixed-methods design that includes both behavioral traces and outcome measures. The paper also contributes a prompt/query taxonomy and evidence on strategic differences (e.g., more follow-ups in AI, more keyword queries in web). However, the significance is currently limited by the lack of evidence for causal exchangeability in the key between-subjects currying comparison, and by the overstatement of 'ease' in the abstract. The paper is exploratory and would benefit from replication with larger samples and better-controlled assignment.

major comments (4)
  1. [Section 5.2.2, 'AI vs. NoAI in Currying'] The currying comparison is an effectively between-subjects comparison with N=16 per group, yet the paper treats it as evidence for a causal effect of the tool. The manuscript states (Section 4.1) that assignment was 'balanced' but does not report random assignment or any balance table comparing the AI and NoAI currying groups on relevant covariates such as prior programming experience, CS1 performance, prior generative-AI familiarity, or motivation. Section 6 does not acknowledge this threat. Please provide a balance table and/or explicitly limit the claim to an associational finding; if causal language is intended, justify exchangeability or adjust for observed covariates.
  2. [Abstract and Section 1] The claim that students 'found it easier to learn a more difficult concept using traditional methods than using ChatGPT' is not directly supported by the measured variables. The study assessed quiz scores and debugging performance; it did not measure perceived ease or difficulty. The 'more difficult' characterization of currying is inferred from overall lower quiz scores and longer completion times, but no treatment-by-difficulty interaction was tested. Please rephrase to describe the actual outcome (e.g., 'participants performed better on a conceptual quiz for the more difficult concept after web search than after ChatGPT'), and soften the causal interpretation.
  3. [Section 5.2.2 and overall statistical reporting] The headline p-value (p = .005) is one of many significance tests performed in the paper (paired t-tests, chi-square tests, GLM, Mann-Whitney U tests) without any multiple-comparison correction. With roughly a dozen tests, a conservative correction would push the threshold to about p < .004, making this result borderline or non-significant. Report the total number of tests, or provide corrected p-values or false-discovery-rate procedures, and include confidence intervals for the effect sizes (e.g., r = 0.50 for the currying comparison).
  4. [Section 5.1.2] The chi-square test for follow-up vs. base queries treats all 237 prompts and 148 queries as independent observations, even though multiple prompts/queries come from the same participant, inducing clustering. The Poisson GLM likewise models per-observation counts without accounting for within-participant correlation. Please use participant-level analyses (e.g., mixed-effects models or per-participant summaries) or explicitly justify the assumption of independence; otherwise these strategy differences may be spurious.
minor comments (4)
  1. [Section 6] Section 6 says 'We randomized the order of tasks and treatments,' but Section 4.1 only says assignment was 'balanced.' Please clarify the exact assignment procedure (randomization vs. counterbalancing) and keep terminology consistent.
  2. [References] The same reference appears twice: [40] and [41] are both Sun et al. (2024), and [47] and [48] are both Younas et al. (2025). Please consolidate duplicate citations.
  3. [Section 4.4.2] The description of the multiple-choice question scoring is ambiguous: 'scored as +1, -1 based on all correct and incorrect answers respectively' could mean all-or-none scoring; please specify the exact scoring rule (e.g., full credit only if all correct options selected and no incorrect options).
  4. [Section 5.1.2] The phrase 'byte-sized answers' appears to be a pun, but it may distract; consider spelling as 'bite-sized' if that is the intended meaning, or use a more standard term.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical comparison whose conclusions are decoupled from its measurement instruments.

full rationale

The paper's derivation chain is empirical rather than formal: participants learn one concept with ChatGPT and the other with traditional web resources, then take a conceptual quiz and a debugging task scored with an explicit rubric (Section 4.5). The central claims—that currying quiz scores were lower in the AI condition than the NoAI condition and that prompts/queries differ by treatment—are supported by standard paired and independent-sample tests (Section 5.2). No parameter is fitted to the outcome and then reported as a prediction; the quiz and debugging instruments were piloted before the study and are not constructed from the treatment outcomes. The coding taxonomy adapts prior published work (Sellen et al., Bolotova et al., Karmaker et al.) but this is methodological borrowing, not circularity: the categories are defined independently of the results. The one reference that includes an author of this paper (Sarkar et al. [35]) is cited in the Discussion as background about challenges of programming with AI, and it is not load-bearing for any quantitative conclusion. The paper itself identifies limitations in Section 6, including small sample size and gender imbalance, and the currying comparison is explicitly acknowledged to rely on small per-cell samples. That is a validity concern (unverified exchangeability of the 16 participants per treatment for the currying task), not a circularity concern, because the reported difference is an observed outcome rather than a quantity forced by how the treatments or measures were defined. No equation reduces to its own input, no fitted value is renamed as a prediction, and no self-citation is used to justify the central claim. Therefore the appropriate finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on the validity of the quiz instrument, the comparability of the small treatment groups, the representativeness of the specific ChatGPT model, and the appropriateness of uncorrected statistical testing. No free-form fitting is used; the only hand-chosen numeric scheme is the quiz scoring rule, which directly shapes the headline result.

free parameters (1)
  • Quiz scoring weights = +1 correct, -1 incorrect, 0 don't know
    Chosen by the authors and documented in Section 4.5. This scoring rule directly determines every quiz score, including the currying AI vs NoAI comparison that is the paper's headline result. Alternative weightings (e.g., no penalty for incorrect answers) could change the outcome.
assumptions (4)
  • domain assumption The conceptual quiz is a valid measure of learning
    Sections 4.4.2 and 5.2.2 use quiz scores as the primary evidence for differential learning outcomes. If the quiz does not track conceptual understanding, the central claim about learning effectiveness collapses.
  • domain assumption Exchangeability of the AI and NoAI groups in the currying comparison
    The headline result compares N=16 participants per treatment for currying. The paper reports balanced task/treatment order but does not demonstrate balance on programming ability, motivation, or prior concept exposure; systematic group differences would bias the treatment effect.
  • domain assumption ChatGPT with GPT 3.5 Turbo represents LLM learning tools generally
    Section 4.1 specifies the model version. The abstract and discussion generalize to 'LLMs such as ChatGPT', so the result is assumed to extend beyond this one model.
  • standard math Statistical inference assumptions hold despite the number of tests
    The paper applies paired t-tests, Mann-Whitney U, chi-square, and a Poisson GLM (Sections 5.1.2 and 5.2). No correction is made for multiple comparisons, which is standard in exploratory work but weakens the evidential value of the single significant result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of To Google or To ChatGPT? A Comparison of CS2 Students' Information Gathering Approaches and Outcomes." pith.science (2026). https://pith.science/paper/2POGTRG3

@misc{pith2026250111935,
  author       = {Pith},
  title        = {Pith review of: To Google or To ChatGPT? A Comparison of CS2 Students' Information Gathering Approaches and Outcomes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2POGTRG3}},
  note         = {Machine review of arXiv:2501.11935}
}
read the original abstract

LLMs such as ChatGPT have been widely adopted by students in higher education as tools for learning programming and related concepts. However, it remains unclear how effective students are and what strategies students use while learning with LLMs. Since the majority of students' experiences in online self-learning have come through using search engines such as Google, evaluating AI tools in this context can help us address these gaps. In this mixed methods research, we conducted an exploratory within-subjects study to understand how CS2 students learn programming concepts using both LLMs as well as traditional online methods such as educational websites and videos to examine how students approach learning within and across both scenarios. We discovered that students found it easier to learn a more difficult concept using traditional methods than using ChatGPT. We also found that students ask fewer follow-ups and use more keyword-based queries for search engines while their prompts to LLMs tend to explicitly ask for information.

Figures

Figures reproduced from arXiv: 2501.11935 by the authors.

Figure 1
Figure 1. Study Protocol (A/B refer to Currying or IIFE; X/Y refer to AI or NoAI) [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Example application of activity analysis codebook (vertical divisions in a segment imply multiple codes applied) [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Percentage of prompts and queries over time [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Debugging Task Performance (LoU refers to lack of understanding) [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Conceptual understanding Quiz Scores that currying is a more difficult topic for participants to learn in the study condition than IIFE in general, or that the post-study quiz in currying task was harder than in IIFE. As a result, we compare the NoAI and AI conditions …
Figure 6
Figure 6. Figure 6: IIFE Debugging Task - Problem Statement .... .... // Start changes below here ! var result ; function adder (){ ( function (a , b ) { result = a + b ; })( a , b ); } adder (); console . log ( result ); [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: IIFE Debugging Task - Expected Solution (Snippet) [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Currying Debugging Task - Problem Statement [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Currying Debugging Task - Expected Solution (Snippet) [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 45 canonical work pages

  1. [1]

    https://www.perplexity.ai

    Perplexity.ai. https://www.perplexity.ai

    work page

  2. [2]

    Student-ai interaction: A case study of cs1 students

    Matin Amoozadeh et al. Student-ai interaction: A case study of cs1 students. In Proceedings of the ACM Conference on International Computing Education Research, 2024. 17

    work page 2024

  3. [3]

    Trust in generative ai among students: An exploratory study

    Matin Amoozadeh et al. Trust in generative ai among students: An exploratory study. Proceedings of the ACM Technical Symposium on Computer Science Education, 2024

    work page 2024

  4. [4]

    AI-enhanced Auto-correction of Programming Exercises: How Effective is GPT-3.5?

    Imen Azaiz, Oliver Deckarm, and Sven Strickroth. AI-enhanced auto-correction of programming exercises: How effective is GPT-3.5? arXiv preprint arXiv:2311.10737, 2023

    work page Pith review arXiv 2023

  5. [5]

    Becker, Paul Denny, James Finnie-Ansley, Andrew Luxton-Reilly, James Prather, and Eddie Antonio Santos

    Brett A. Becker, Paul Denny, James Finnie-Ansley, Andrew Luxton-Reilly, James Prather, and Eddie Antonio Santos. Programming is hard - or at least it used to be: Educational opportunities and challenges of ai code generation. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, SIGCSE 2023, page 500–506, New York, NY, USA...

    work page 2023

  6. [6]

    Bruce Croft, and Mark Sanderson

    Valeriia Bolotova, Vladislav Blinov, Falk Scholer, W. Bruce Croft, and Mark Sanderson. A non-factoid question-answering taxonomy. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval , SIGIR ’22, page 1196–1207, New York, NY, USA, 2022. Association for Computing Machinery

    work page 2022

  7. [7]

    Two studies of opportunistic programming: interleaving web foraging, learning, and writing code

    Joel Brandt, Philip J Guo, Joel Lewenstein, Mira Dontcheva, and Scott R Klemmer. Two studies of opportunistic programming: interleaving web foraging, learning, and writing code. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , pages 1589–1598. ACM, 2009

    work page 2009

  8. [8]

    A coefficient of agreement for nominal scales

    Jacob Cohen. A coefficient of agreement for nominal scales. Educational and Psychological Measurement , 20(1):37–46, 1960

    1960

Show all 50 references
  1. [9]

    Can we trust ai-generated educational content? comparative analysis of human and ai-generated learning resources

    Paul Denny et al. Can we trust ai-generated educational content? comparative analysis of human and ai-generated learning resources. arXiv preprint arXiv:2306.10509, 2023

    2023 arXiv

  2. [10]

    Robillard

    Ekwa Duala-Ekoko and Martin P. Robillard. The information gathering strategies of api learners. In Technical Report. School of Computer Science, McGill University, 2014

    2014

  3. [11]

    Immediately invoked function expressions (iife) in javascript

    Geeks for Geeks. Immediately invoked function expressions (iife) in javascript. https://www.geeksforgeeks.org/immediately-invoked-function- expressions-iife-in-javascript/, 2024

    2024

  4. [12]

    Six opportunities for scientists and engineers to learn programming using ai tools such as chatgpt

    Philip J Guo. Six opportunities for scientists and engineers to learn programming using ai tools such as chatgpt. IEEE Computing in Science & Engineering, 2023

    2023

  5. [13]

    Currying in javascript explained with examples

    Built In. Currying in javascript explained with examples. https://builtin.com/software-engineering-perspectives/currying-javascript, 2023

    2023

  6. [14]

    TELeR: A general taxonomy of LLM prompts for benchmarking complex tasks

    Shubhra Kanti Karmaker Santu and Dongji Feng. TELeR: A general taxonomy of LLM prompts for benchmarking complex tasks. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023 , pages 14197–14203, Singapore, December

    2023

  7. [15]

    How novices use llm-based code generators to solve cs1 coding tasks in a self-paced learning environment

    Majeed Kazemitabaar et al. How novices use llm-based code generators to solve cs1 coding tasks in a self-paced learning environment. In Koli Calling International Conference on Computing Education Research . ACM, 2023

    2023

  8. [16]

    Codeaid: Evaluating a classroom deployment of an llm-based programming assistant that balances student and educator needs

    Majeed Kazemitabaar, Runlong Ye, Xiaoning Wang, Austin Zachary Henley, Paul Denny, Michelle Craig, and Tovi Grossman. Codeaid: Evaluating a classroom deployment of an llm-based programming assistant that balances student and educator needs. In Proceedings of the 2024 CHI Confe...

    2024

  9. [17]

    Information needs in collocated software development teams

    Andrew J Ko, Robert DeLine, and Gina Venolia. Information needs in collocated software development teams. In 29th International Conference on Software Engineering (ICSE’07), pages 344–353. IEEE, 2007

    2007

  10. [18]

    Multimodality of ai for education: Towards artificial general intelligence, 2023

    Gyeong-Geon Lee, Lehong Shi, Ehsan Latif, Yizhu Gao, Arne Bewersdorff, Matthew Nyaaba, Shuchen Guo, Zihao Wu, Zhengliang Liu, Hui Wang, Gengchen Mai, Tiaming Liu, and Xiaoming Zhai. Multimodality of ai for education: Towards artificial general intelligence, 2023

    2023

  11. [19]

    The impact of generative ai on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers

    Hao-Ping Lee, Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, and Nicholas Wilson. The impact of generative ai on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. In Proceedings ...

    2025

  12. [20]

    Comparing code explanations created by students and large language models

    Juho Leinonen, Paul Denny, Stephen MacNeil, Sami Sarsa, Seth Bernstein, Joanne Kim, Andrew Tran, and Arto Hellas. Comparing code explanations created by students and large language models. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science E...

    2023

  13. [21]

    Experiences from using code explanations generated by large language models in a web software development e-book

    Stephen MacNeil, Andrew Tran, Arto Hellas, Joanne Kim, Sami Sarsa, Paul Denny, Seth Bernstein, and Juho Leinonen. Experiences from using code explanations generated by large language models in a web software development e-book. In Proceedings of the 54th ACM Technical Symposiu...

    2023

  14. [22]

    Why computing students learn on their own: Motivation for self-directed learning of computing

    Robert McCartney, Jonas Boustedt, Anna Eckerdal, Kate Sanders, Lynda Thomas, Jan Erik Moström, and Carol Zander. Why computing students learn on their own: Motivation for self-directed learning of computing. ACM Transactions on Computing Education (TOCE) , 16(1):1–18, 2016

    2016

  15. [23]

    Computing students learning computing informally

    Robert Mccartney, Anna Eckerdal, Jan Erik Moström, Kate Sanders, Lynda Thomas, and Carol Zander. Computing students learning computing informally. In European Conference on Modelling and Simulation , 2010

    2010

  16. [24]

    Social media as a complementary learning tool for teaching and learning: The case of youtube

    Sedigheh Moghavvemi, Ainin Sulaiman, Noor Ismawati Jaafar, and Nafisa Kasem. Social media as a complementary learning tool for teaching and learning: The case of youtube. The International Journal of Management Education , 16(1):37–42, 2018

    2018

  17. [25]

    Silvia Muller, Monica Babes-Vroman, Mary Emenike, and Thu D. Nguyen. Exploring novice programmers’ homework practices: Initial observations of information seeking behaviors. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education , SIGCSE ’20, page 333...

    2020

  18. [26]

    The impact of prompt engineering and a generative ai-driven tool on autonomous learning: A case study

    Kovan Mzwri and Márta Turcsányi-Szabo. The impact of prompt engineering and a generative ai-driven tool on autonomous learning: A case study. Education Sciences, 15(2):199, 2025

    2025

  19. [27]

    How beginning programmers and code llms (mis)read each other

    Sydney Nguyen et al. How beginning programmers and code llms (mis)read each other. In Proceedings of the CHI Conference on Human Factors in Computing Systems, 2024. 18

    2024

  20. [28]

    Information foraging in information access environments

    Peter Pirolli and Stuart Card. Information foraging in information access environments. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 51–58, 1995

    1995

  21. [29]

    The robots are here: Navigating the generative ai revolution in computing education

    James Prather, Paul Denny, Juho Leinonen, Brett A Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton-Reilly, et al. The robots are here: Navigating the generative ai revolution in computing education. arXiv preprint arXiv:2310.0...

    arXiv

  22. [30]

    The widening gap: The benefits and harms of generative ai for novice programmers

    James Prather et al. The widening gap: The benefits and harms of generative ai for novice programmers. In ICER ’24. ACM, 2024

    2024

  23. [31]

    Randrianasolo, Brett Becker, Bailey Kimmel, Jared Wright, and Ben Briggs

    James Prather, Brent Reeves, Juho Leinonen, Stephen MacNeil, Arisoa S. Randrianasolo, Brett Becker, Bailey Kimmel, Jared Wright, and Ben Briggs. The widening gap: The benefits and harms of generative ai for novice programmers, 2024

    2024

  24. [32]

    it’s weird that it knows what i want

    James Prather, Brent N. Reeves, Paul Denny, Brett A. Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, and Eddie Antonio Santos. "it’s weird that it knows what i want": Usability and interactions with copilot for novice programmers. arXiv prepri...

    2023 arXiv

  25. [33]

    Chatgpt in computer science curriculum assessment: An analysis of its successes and shortcomings

    Basit Qureshi. Chatgpt in computer science curriculum assessment: An analysis of its successes and shortcomings. In 2023 9th International Conference on e-Society, e-Learning and e-Technologies, ICSLT 2023, page 7–13. ACM, June 2023

    2023

  26. [34]

    Next-step hint generation for introductory programming using large language models, 2023

    Lianne Roest, Hieke Keuning, and Johan Jeuring. Next-step hint generation for introductory programming using large language models, 2023

    2023

  27. [35]

    Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn

    Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn. What is it like to program with artificial intelligence?, 2022

    2022

  28. [36]

    How knowledge workers use the web

    Abigail J Sellen, Rachel Murphy, and Kate L Shaw. How knowledge workers use the web. Proceedings of the SIGCHI conference on Human factors in computing systems, pages 227–234, 2002

    2002

  29. [37]

    Sellen, Rachel Murphy, and Kate L

    Abigail J. Sellen, Rachel Murphy, and Kate L. Shaw. How knowledge workers use the web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , CHI ’02, page 227–234, New York, NY, USA, 2002. Association for Computing Machinery

    2002

  30. [38]

    An investigation of the drivers of novice programmers’ intentions to use web search and genai

    James Skripchuk, John Bacher, and Thomas Price. An investigation of the drivers of novice programmers’ intentions to use web search and genai. In Proceedings of the 2024 ACM Conference on International Computing Education Research - Volume 1 , ICER ’24, page 487–501, New York,...

    2024

  31. [39]

    Stack overflow developer survey 2024, 2024

    Stack Overflow. Stack overflow developer survey 2024, 2024

    2024

  32. [40]

    Investigating students’ programming behaviors, interaction qualities and perceptions through prompt-based learning in ChatGPT

    Dan Sun, Azzeddine Boudouaia, Junfeng Yang, and Jie Xu. Investigating students’ programming behaviors, interaction qualities and perceptions through prompt-based learning in ChatGPT. Humanities and Social Sciences Communications , 11:1447, 2024

    2024

  33. [41]

    Investigating students’ programming behaviors, interaction qualities and perceptions through prompt-based learning in chatgpt

    Dan Sun, Azzeddine Boudouaia, Junfeng Yang, and Jie Xu. Investigating students’ programming behaviors, interaction qualities and perceptions through prompt-based learning in chatgpt. Humanities and Social Sciences Communications , 11:1447, 2024

    2024

  34. [42]

    Zhen Sun, Yuchen Liu, and David J. Malan. Prompt learning for programming education with ChatGPT. In Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 . ACM, 2024

    2024

  35. [43]

    Comparing non-visual and visual information foraging on the web

    Pooja Upadhyay. Comparing non-visual and visual information foraging on the web. In CHI’20 Extended Abstracts, pages 1–8. ACM, 2020

    2020

  36. [44]

    Fostering postgraduate student engagement: online resources supporting self-directed learning in a diverse cohort

    Mello Luciane V. Fostering postgraduate student engagement: online resources supporting self-directed learning in a diverse cohort. Research in Learning Technology, 24, Mar. 2016

    2016

  37. [45]

    The rise and development of digital education

    Martin Weller. The rise and development of digital education. In Handbook of open, distance and digital education , pages 1–17. Springer, 2022

    2022

  38. [46]

    K. R. Wirth, C. du Toit-Brits, E. Vos, J. H. Blignaut, J. Olivier, S. Mahlaba, others, and G. van Tonder.Self-Directed Learning: Curriculum implementation, praxis and scholarship in context . 2024

    2024

  39. [47]

    A comprehensive systematic review of ai-driven approaches to self-directed learning

    Muhammad Younas, Dina Abdel Salam El-Dakhs, and Yicun Jiang. A comprehensive systematic review of ai-driven approaches to self-directed learning. IEEE Access, 13:38387–38403, 2025

    2025

  40. [48]

    A comprehensive systematic review of ai-driven approaches to self-directed learning

    Muhammad Younas, Dina Abdel Salam El-Dakhs, and Yicun Jiang. A comprehensive systematic review of ai-driven approaches to self-directed learning. IEEE Access, 13:38387–38396, 2025. 19

    2025

  41. [2023]

    Association for Computational Linguistics

  42. [2024]

    Association for Computing Machinery

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.