pith. sign in

arxiv: 2602.07142 · v2 · submitted 2026-02-06 · 💻 cs.HC · cs.AI

Exploring Teachers' Perspectives on Using Conversational AI Agents for Group Collaboration

Prerna Ravi , Car\'umey Stevens , Beatriz Flamia Azevedo , Jasmine David , Brandon Hanks , Hal Abelson , Grace Lin , Emma Anderson This is my paper

Pith reviewed 2026-05-16 06:00 UTC · model grok-4.3

classification 💻 cs.HC cs.AI
keywords conversational AIgroup collaborationK-12 educationteacher perspectivesAI in educationvoice-based agentspedagogical designclassroom technology
0
0 comments X

The pith

Teachers see a voice-based AI agent as a useful spark for classroom group engagement but raise concerns about autonomy, trust, and fit with teaching goals.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper reports an exploratory study where 33 K-12 teachers tested Phoenix, a voice-based conversational agent built to act as a near-peer during in-person group work. Teachers interacted with the agent in playtesting sessions and then shared their views through surveys and focus groups. While many noted that the agent could prompt more active student participation, they also flagged worries about students losing independence, placing too much trust in the AI, its human-like traits, and whether its behavior aligned with classroom teaching practices. The work surfaces teachers' mental models of AI in education and identifies design tensions that future group-facing agents will need to navigate.

Core claim

Teachers appreciated Phoenix's capacity to stimulate engagement in face-to-face groups but expressed concerns around autonomy, trust, anthropomorphism, and pedagogical alignment, yielding empirical insights into educators' mental models of AI and core design tensions for group-facing agents.

What carries the argument

Phoenix, a voice-based conversational agent designed to function as a near-peer mediator in face-to-face group collaboration, analyzed through qualitative data from teacher playtesting sessions, surveys, and focus groups.

Load-bearing premise

That the perceptions collected from 33 teachers during controlled playtesting sessions, surveys, and focus groups accurately reflect how the agent would perform and be received in everyday classroom settings with diverse student groups.

What would settle it

Deploy Phoenix in ordinary K-12 classrooms for several weeks and compare observed student engagement levels, instances of student autonomy, and teacher-reported trust issues against matched groups that do not use the agent.

read the original abstract

Collaboration is a cornerstone of 21st-century learning, yet teachers continue to face challenges in supporting productive peer interaction. Emerging generative AI tools offer new possibilities for scaffolding collaboration, but their role in mediating in-person group work remains underexplored, especially from the perspective of educators. This paper presents findings from an exploratory qualitative study with 33 K12 teachers who interacted with Phoenix, a voice-based conversational agent designed to function as a near-peer in face-to-face group collaboration. Drawing on playtesting sessions, surveys, and focus groups, we examine how teachers perceived the agent's behavior, its influence on group dynamics, and its classroom potential. While many appreciated Phoenix's capacity to stimulate engagement, they also expressed concerns around autonomy, trust, anthropomorphism, and pedagogical alignment. We contribute empirical insights into teachers' mental models of AI, reveal core design tensions, and outline considerations for group-facing AI agents that support meaningful, collaborative learning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper presents findings from an exploratory qualitative study involving 33 K-12 teachers who interacted with Phoenix, a voice-based conversational AI agent designed to act as a near-peer in face-to-face group collaboration. Data were collected via playtesting sessions, surveys, and focus groups to examine teachers' perceptions of the agent's behavior, its influence on group dynamics, and its classroom potential. Key results indicate that many teachers appreciated the agent's ability to stimulate engagement, while also raising concerns about autonomy, trust, anthropomorphism, and pedagogical alignment. The authors contribute empirical insights into teachers' mental models of AI, identify core design tensions, and outline considerations for developing group-facing AI agents to support collaborative learning.

Significance. If the results hold, this work provides timely empirical data on educator perspectives regarding AI tools for mediating in-person collaboration, an area that remains underexplored in HCI and educational technology. The multi-method qualitative approach yields rich insights into design tensions such as trust and anthropomorphism that can inform future agent development. Strengths include direct engagement with practicing teachers and the identification of actionable considerations for group-facing agents. The exploratory framing appropriately tempers broad claims, though the controlled nature of the data collection constrains immediate applicability to authentic classroom environments.

major comments (2)
  1. [Methods] Methods section: The description of the qualitative analysis process (how themes were derived from playtesting sessions, surveys, and focus groups) lacks sufficient detail on coding procedures, theme identification, or measures to ensure rigor such as inter-coder reliability. This is load-bearing for the central claims about specific concerns (autonomy, trust, anthropomorphism, pedagogical alignment) and makes it difficult to evaluate whether the themes were derived systematically or influenced by researcher bias.
  2. [Discussion] Discussion / Abstract: The claims about the agent's 'classroom potential' and design considerations for group-facing agents rest on perceptions gathered exclusively from controlled playtesting sessions without students present, curriculum constraints, or repeated real-world exposure. The untested assumption that these artificial-session views predict authentic classroom reception and performance needs stronger qualification or additional evidence to support the paper's broader implications.
minor comments (1)
  1. [Abstract] Abstract: The abstract could briefly note the exploratory and controlled nature of the study to better set reader expectations for the scope of the findings.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback, which helps us strengthen the transparency and framing of our exploratory study. We address each major comment below, indicating revisions to the manuscript where appropriate.

read point-by-point responses

  1. Referee: [Methods] Methods section: The description of the qualitative analysis process (how themes were derived from playtesting sessions, surveys, and focus groups) lacks sufficient detail on coding procedures, theme identification, or measures to ensure rigor such as inter-coder reliability. This is load-bearing for the central claims about specific concerns (autonomy, trust, anthropomorphism, pedagogical alignment) and makes it difficult to evaluate whether the themes were derived systematically or influenced by researcher bias.

    Authors: We agree that greater detail on the analysis process is warranted to support evaluation of the themes. In the revised manuscript, we will expand the Methods section to describe the thematic analysis in full: an inductive, iterative coding process applied across playtesting transcripts, open-ended survey responses, and focus group recordings; the steps for initial code generation, clustering into themes, and refinement through team discussion; and measures for rigor including dual independent coding of a data subset by two researchers followed by consensus resolution of discrepancies. This addition will clarify the systematic derivation of the reported concerns without altering the exploratory nature of the work. revision: yes

  2. Referee: [Discussion] Discussion / Abstract: The claims about the agent's 'classroom potential' and design considerations for group-facing agents rest on perceptions gathered exclusively from controlled playtesting sessions without students present, curriculum constraints, or repeated real-world exposure. The untested assumption that these artificial-session views predict authentic classroom reception and performance needs stronger qualification or additional evidence to support the paper's broader implications.

    Authors: We acknowledge the controlled setting of the playtesting sessions and the resulting limits on direct claims about classroom performance. As an explicitly exploratory study, our contributions center on surfacing teachers' perceptions and design tensions rather than validated predictions. We will revise the Discussion to more explicitly qualify these boundaries, stressing the artificial conditions and the necessity of future in-classroom validation studies. We will also adjust phrasing in the Abstract and Discussion to frame 'classroom potential' and design considerations as preliminary insights intended to guide subsequent development and research, thereby aligning claims more tightly with the data collected. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical qualitative study with direct data collection

full rationale

The paper reports an exploratory qualitative study drawing on playtesting sessions, surveys, and focus groups with 33 K12 teachers interacting with the Phoenix agent. No equations, fitted parameters, derivations, or predictions exist that could reduce by construction to inputs or self-citations. Central claims about teachers' perceptions of engagement, autonomy, trust, anthropomorphism, and pedagogical alignment are presented as direct empirical observations rather than derived quantities. The analysis is self-contained against external benchmarks with no load-bearing self-citation chains or ansatz smuggling.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 1 invented entities

The central claims rest on the assumption that the designed agent Phoenix functions as intended during sessions and that teacher self-reports in a research setting capture authentic classroom concerns. No free parameters or formal axioms are present; the invented entity is the agent itself.

invented entities (1)
  • Phoenix no independent evidence
    purpose: Voice-based conversational agent acting as a near-peer to scaffold face-to-face group collaboration
    The agent is introduced as the intervention whose behavior and effects are evaluated; no independent evidence of its performance outside the study sessions is provided.

pith-pipeline@v0.9.0 · 5478 in / 1243 out tokens · 37497 ms · 2026-05-16T06:00:48.713352+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

36 extracted references · 36 canonical work pages

  1. [1]

    npj Science of Learning10(1), 1 (2025)

    Ackermann, H., Henke, A., Chevalère, J., Yun, H.S., Hafner, V.V., Pinkwart, N., Lazarides, R.: Physical embodiment and anthropomorphism of ai tutors and their role in student enjoyment and performance. npj Science of Learning10(1), 1 (2025)

    work page 2025

  2. [2]

    Thinking Skills and creativity27, 78–91 (2018)

    Al-Samarraie, H., Hurmuzan, S.: A review of brainstorming techniques in higher education. Thinking Skills and creativity27, 78–91 (2018)

    work page 2018

  3. [3]

    Amiot, C., Charoy, F., Dinet, J.: Chatbots in collaborative settings and their im- pact on virtual teamwork. Proc. ACM Hum.-Comput. Interact.9(2), 1–18 (2025)

    work page 2025

  4. [4]

    Anderson, E., Lin, G.C., Farid, A., Fenech, M., Hanks, B., Klopfer, E., Doherty, E., Hirshfield, L., Ko, M.L.M., Foltz, P., et al.: Exploring genai technologies within collaborative learning. In: Proceedings of the 18th International Conference on Teachers’ Perspectives on Conversational AI Agents for Collaboration 13 Computer-Supported Collaborative Lear...

    work page 2025

  5. [5]

    International Journal of Artificial Intelligence in Education pp

    de Araujo, A., Papadopoulos, P.M., McKenney, S., de Jong, T.: Investigating the impact of a collaborative conversational agent on dialogue productivity and knowl- edge acquisition. International Journal of Artificial Intelligence in Education pp. 1–27 (2025)

    work page 2025

  6. [6]

    Qualitative re- search in sport, exercise and health11(4), 589–597 (2019)

    Braun, V., Clarke, V.: Reflecting on reflexive thematic analysis. Qualitative re- search in sport, exercise and health11(4), 589–597 (2019)

    work page 2019

  7. [7]

    Computers & Education53(4), 1147–1154 (2009)

    Casamayor, A., Amandi, A., Campo, M.: Intelligent assistance for teachers in collaborative e-learning environments. Computers & Education53(4), 1147–1154 (2009)

    work page 2009

  8. [8]

    In: Proceedings of the 29th International Conference on Intelligent User Interfaces (IUI ’24)

    Chiang, C.W., Lu, Z., Li, Z., Yin, M.: Enhancing ai-assisted group decision making through llm-powered devil’s advocate. In: Proceedings of the 29th International Conference on Intelligent User Interfaces (IUI ’24). pp. 103–119. Association for Computing Machinery, New York, NY, USA (2024)

    work page 2024

  9. [9]

    Dyke, G., Adamson, D., Howley, I., Rosé, C.P.: Enhancing scientific reasoning and discussionwithconversationalagents.IEEETransactionsonLearningTechnologies 6(3), 240–247 (2013)

    work page 2013

  10. [10]

    British Journal of Educational Technology56(2), 712–733 (2025)

    Edwards, J., Nguyen, A., Lämsä, J., Sobocinski, M., Whitehead, R., Dang, B., Roberts, A.S., Järvelä, S.: Human-ai collaboration: Designing artificial agents to facilitate socially shared regulation among learners. British Journal of Educational Technology56(2), 712–733 (2025)

    work page 2025

  11. [11]

    21st century skills

    Evans, C.M.: Measuring student success skills: A review of the literature on col- laboration. 21st century skills. Tech. rep., National Center for the Improvement of Educational Assessment (2020)

    work page 2020

  12. [12]

    Computers & Education 138, 1–12 (2019)

    Hernández-Sellés, N., Muñoz-Carril, P.C., González-Sanmamed, M.: Computer- supported collaborative learning: An analysis of the relationship between interac- tion, emotional support and online collaborative tools. Computers & Education 138, 1–12 (2019)

    work page 2019

  13. [13]

    In: Proceedings of the 30th International Conference on Intelligent User Interfaces

    Houde, S., Brimijoin, K., Muller, M., Ross, S.I., Silva Moran, D.A., Gonzalez, G.E., Kunde, S., Foreman, M.A., Weisz, J.D.: Controlling ai agent participation in group conversations: A human-centered approach. In: Proceedings of the 30th International Conference on Intelligent User Interfaces. pp. 390–408 (2025)

    work page 2025

  14. [14]

    Prentice Hall, Englewood Cliffs, NJ, 5th ed

    Johnson, D.W., Johnson, F.P.: Joining Together: Group Theory and Group Skills. Prentice Hall, Englewood Cliffs, NJ, 5th ed. edn. (1991)

    work page 1991

  15. [15]

    Proceedings of the ACM on human-computer interaction2(CSCW), 1–31 (2018)

    Lau, J., Zimmerman, B., Schaub, F.: Alexa, are you listening? privacy perceptions, concerns and privacy-seeking behaviors with smart speakers. Proceedings of the ACM on human-computer interaction2(CSCW), 1–31 (2018)

    work page 2018

  16. [16]

    Leong, J., Tang, J., Cutrell, E., Junuzovic, S., Baribault, G.P., Inkpen, K.: Dittos: Personalized, embodied agents that participate in meetings when you are unavail- able. Proc. ACM Hum.-Comput. Interact.8(CSCW2), 1–28 (2024)

    work page 2024

  17. [17]

    In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

    Liu, J., Yao, Y., An, P., Wang, Q.: Peergpt: Probing the roles of llm-based peer agents as team moderators and participants in children’s collaborative learning. In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA (2024)

    work page 2024

  18. [18]

    Education and Information Technologies pp

    Lu, G., Ba, S., Yang, L.: Optimizing gai-assisted formative feedback: an experi- mental study on its effects on engagement, shared metacognition, and learning per- formance in online collaborative learning. Education and Information Technologies pp. 1–31 (2025) 14 P. Ravi et al

    work page 2025

  19. [19]

    preprint arXiv:2601.17962 (2026)

    Lyu, W., Wang, Y., Yue, M., Sun, Y., Suh, J., Kier, M., Yao, Z., Zhang, Y.: Designing ai peers for collaborative mathematical problem solving with middle school students: A participatory design study. preprint arXiv:2601.17962 (2026)

    work page arXiv 2026

  20. [20]

    In: Proceedings of the Human Factors and Er- gonomics Society Annual Meeting

    Mosier, K.L., Skitka, L.J., Burdick, M.D., Heers, S.T.: Automation bias, account- ability, and verification behaviors. In: Proceedings of the Human Factors and Er- gonomics Society Annual Meeting. vol. 40, pp. 204–208. SAGE Publications Sage CA: Los Angeles, CA (1996)

    work page 1996

  21. [21]

    In: Proceedings of the SIGCHI conference on Human factors in computing systems

    Nass, C., Steuer, J., Tauber, E.R.: Computers are social actors. In: Proceedings of the SIGCHI conference on Human factors in computing systems. pp. 72–78 (1994)

    work page 1994

  22. [22]

    Digital1(1), 18–33 (2021)

    Pérez-Marín, D.: A review of the practical applications of pedagogic conversational agents to be used in school and university classrooms. Digital1(1), 18–33 (2021)

    work page 2021

  23. [23]

    International Journal of Human-Computer Studies195, 103407 (2025)

    Piercy, C.W., Montgomery-Vestecka, G., Lee, S.K.: Gender and accent stereotypes in communication with an intelligent virtual assistant. International Journal of Human-Computer Studies195, 103407 (2025)

    work page 2025

  24. [24]

    PROJECT (Professional Journal of English Education)2(5), 594 (2019)

    Rahmayanti, P., Saraswati, P.A., Bhuana, G.P., et al.: The use of ice breaker to improve students’ motivation in learning english at the tenth grade students of smk ypkkp. PROJECT (Professional Journal of English Education)2(5), 594 (2019)

    work page 2019

  25. [25]

    Journal of Educational Technology & Society10(4), 257–268 (2007), retrieved August 28, 2025 from https://www.learntechlib.org/p/74872/

    Roberts, T.S., McInnerney, J.M.: Seven problems of online group learning (and their solutions). Journal of Educational Technology & Society10(4), 257–268 (2007), retrieved August 28, 2025 from https://www.learntechlib.org/p/74872/

    work page 2007

  26. [26]

    In: International Conference on Artificial Intelligence in Education

    Sankaranarayanan, S., Kandimalla, S.R., Hasan, S., An, H., Bogart, C., Murray, R.C., Hilton, M., Sakr, M., Rosé, C.: Agent-in-the-loop: Conversational agent support in service of reflection for learning during collaborative programming. In: International Conference on Artificial Intelligence in Education. pp. 273–278. Springer (2020)

    work page 2020

  27. [27]

    Cathcart Press (2017)

    Shaffer, D.W.: Quantitative Ethnography. Cathcart Press (2017)

    work page 2017

  28. [28]

    In: Proceedings of the 6th ACM Conference on Conversational User Interfaces

    Sun, G., Zhan, X., Such, J.: Building better ai agents: A provocation on the util- isation of persona in llm-based conversational agents. In: Proceedings of the 6th ACM Conference on Conversational User Interfaces. pp. 1–6 (2024)

    work page 2024

  29. [29]

    Computers and Edu- cation: Artificial Intelligence3, 100097 (2022)

    Tan, S.C., Lee, A.V.Y., Lee, M.: A systematic review of artificial intelligence tech- niques for collaborative learning over the past two decades. Computers and Edu- cation: Artificial Intelligence3, 100097 (2022)

    work page 2022

  30. [30]

    Pedagogies: An International Journal19(3), 493–503 (2024)

    Tang, K.S., Cooper, G., Rappa, N., Cooper, M., Sims, C., Nonis, K.: A dialogic approach to transform teaching, learning & assessment with generative ai in sec- ondary education: A proof of concept. Pedagogies: An International Journal19(3), 493–503 (2024)

    work page 2024

  31. [31]

    Zawacki-Richter, O., Marín, V.I., Bond, M., Gouverneur, F.: Systematic review of research on artificial intelligence applications in higher education–where are the educators? International journal of educational technology in higher education 16(1), 1–27 (2019)

    work page 2019

  32. [32]

    Educational Psychology Review37(4), 106 (2025)

    Zhang, H., Liu, Y., Jiang, M., Chen, J., Wang, M., Paas, F.: Emotional artifi- cial intelligence in education: A systematic review and meta-analysis. Educational Psychology Review37(4), 106 (2025)

    work page 2025

  33. [33]

    Zhang, R., Duan, W., Flathmann, C., McNeese, N., Freeman, G., Williams, A.: Investigating ai teammate communication strategies and their impact in human- ai teams for effective teamwork. Proc. ACM Hum.-Comput. Interact.7(CSCW2) (2023)

    work page 2023

  34. [34]

    visual: How humans perceive and collaborate with ai teammates using different communication modalities in various human-ai team compositions

    Zhang, R., Duan, W., Flathmann, C., McNeese, N., Knijnenburg, B., Freeman, G.: Verbal vs. visual: How humans perceive and collaborate with ai teammates using different communication modalities in various human-ai team compositions. Proc. ACM Hum.-Comput. Interact.8(CSCW2) (Nov 2024) Teachers’ Perspectives on Conversational AI Agents for Collaboration 15

    work page 2024

  35. [35]

    In: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems

    Zhang, Z., Sun, B., An, P.: Breaking barriers or building dependency? exploring team-llm collaboration in ai-infused classroom debate. In: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. CHI ’25, Association for Computing Machinery, New York, NY, USA (2025)

    work page 2025

  36. [36]

    In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

    Zheng, C., Wu, Y., Shi, C., Ma, S., Luo, J., Ma, X.: Competent but rigid: Identify- ing the gap in empowering ai to participate equally in group decision-making. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. pp. 1–19 (2023)

    work page 2023