{"id":"e38de8bc-c92a-4d9f-94a9-7f43d94eb243","arxiv_id":"2505.17557","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Novobo, a teachable AI agent that acts as an apprentice teacher, helped 30 teachers in 10 sessions externalize and co-construct knowledge about instructional gestures through group discussion and embodied demonstration.","lead":"A new AI system, Novobo, has teachers work in small groups to teach a virtual mentee how to use instructional gestures. In a study with 30 teachers, this peer-learning setup prompted teachers to share and refine their gesture knowledge together.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No control condition supports the causal claim that Novobo's design—rather than small-group discussion itself—prompted knowledge exchange, and 'internalization' is inferred without any post-session measure.","rationale":"The paper is a well-executed qualitative study with rich observational evidence, and the vignette analysis is appropriate for documenting in-context knowledge exchange. The reader correctly identifies the causal attribution gap: Section 4.2 lacks any comparison condition, and teacher study groups naturally discuss gestures; the four-stage procedure may itself produce the observed behaviors. The same gap extends to 'internalization', which cannot be established from session data alone. Because the design contribution and descriptive findings remain plausible and useful, the appropriate verdict remains conditional acceptance with a request to soften the causal wording or add baseline evidence—not rejection. The proposed no-system control directly tests the load-bearing assumption and would settle whether the observed peer learning is attributable to Novobo's specific design features.","tokens_in":26916,"tokens_out":2735,"duration_ms":24078,"concrete_test":"Run a minimal control condition: recruit comparable same-school teacher groups (n=10) for a 45-minute workshop with the same teaching scenarios and the same four discussion prompts (rate, comment, demonstrate, explain), but with a human facilitator or static worksheet instead of Novobo; apply the same SECI vignette coding to both arms and compare the frequency and depth of externalization, exchange, and co-construction episodes. If the no-system arm shows equivalent episodes, the causal attribution to Novobo's design features is unsupported; if the Novobo arm shows substantially more or qualitatively different episodes, the claim is strengthened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the abstract is that 'Novobo prompted teachers to share tacit knowledge' and 'helped teachers externalize, exchange, and internalize their embodied knowledge.' The load-bearing step is causal attribution of observed peer learning to Novobo's design (mentee positioning, skeletal mirror, structured four-stage flow). The study has no baseline or control condition: Section 4.2 sends all 10 groups through the same 45-minute four-stage interaction, and the researcher-provided instructions themselves require teachers to rate, comment, demonstrate, explain, and reach consensus for the system. Thus the externalization, socialization, and combination episodes coded in Section 5 could be produced by the structured group activity rather than by the agent's mentee role or skeletal mirror. Additionally, 'internalization' is claimed even though no post-session measure of gesture use, retention, or transfer is reported; all evidence is in-session observation and focus-group self-report. The paper's stated limitations (Section 7.6) do not acknowledge this attribution gap. This does not invalidate the design contribution or the descriptive qualitative findings, but the abstract's causal wording exceeds the evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Novobo, a teachable AI-agent that acts as a mentee for teachers' peer learning of instructional gestures. The system combines a large-language-model conversation agent with Retrieval-Augmented Generation from a curated gesture knowledge base, and a skeletal-mirror interface for embodied practice. The authors report a collaborative design process with six teachers, and an evaluation with 30 teachers in 10 group sessions, each following a four-stage interaction loop of question posing, commentary, demonstration, and explanation. Qualitative analysis of video vignettes and focus-group interviews is framed through the SECI knowledge-creation model. The central claim is that Novobo 'prompted teachers to share tacit knowledge through conversation and movement' and helped them 'externalize, exchange, and internalize' embodied knowledge.","tokens_in":27121,"tokens_out":2781,"duration_ms":24201,"significance":"The work opens a useful design space: applying teachable agents to teachers' professional development rather than to students, and targeting instructional gestures as embodied, tacit knowledge. The design is responsibly grounded: knowledge sources are traceable, the skeletal mirror is a thoughtful response to self-consciousness, and the participatory iteration with teachers appears genuine. The qualitative material is rich and well-coded, with concrete vignettes that give credibility to the descriptive findings. If recast as an exploratory, descriptive study, the paper would be a solid contribution to HCI and teacher professional development. However, the abstract and conclusion make causal claims about Novobo's design features that the single-arm evaluation design cannot support, and the 'internalization' claim lacks an appropriate measurement. The significance of the contribution is therefore conditional on substantial revision of the claims.","major_comments":[{"comment":"The abstract states that 'Novobo prompted teachers to share tacit knowledge' and 'helped teachers externalize, exchange, and internalize' knowledge, but the evaluation contains no control or baseline condition. In Section 4.2, all 10 groups follow the same 45-minute four-stage protocol, and the researcher-provided instructions themselves require teachers to rate, comment, demonstrate, explain, and reach consensus. The observed externalization, socialization, and combination episodes in Section 5 could therefore be produced by the structured group task rather than by Novobo's mentee positioning or skeletal mirror. The causal wording in the abstract and in Section 7.1 overstates the evidence. I recommend reframing the contribution as a descriptive, exploratory study of a design concept, or adding a comparison condition that isolates the system's features.","section":"§4.2, §5, abstract"},{"comment":"The paper claims that teachers 'internalized' embodied knowledge, yet there is no post-session measure of gesture use, retention, or transfer. All evidence is in-session observation and focus-group self-report. In Section 5.4, the process labeled 'internalization' is actually the act of entering a consensus wording into Novobo, which is closer to SECI's 'combination' than to 'internalization.' To support an internalization claim, the study would need a delayed follow-up, a classroom-observation measure, or some other behavioral transfer indicator. Otherwise the claim should be removed or clearly relabeled as 'combination' or 'in-session co-construction.'","section":"§5.4, Figure 4, §4.3"},{"comment":"Several interpretive statements attribute observed or reported effects to specific design features without any manipulation of those features. For example, Section 5.2 says 'This openness may stem from the mentor-mentee dynamics,' and Section 6.3 concludes that the mentee narrative 'reduced peer pressure' based on focus-group comments. Because every session used the same mentee framing, the data cannot distinguish the effect of that framing from the effect of any other group-discussion context. The qualitative self-reports are informative as user perceptions, but the causal explanations should be phrased as hypotheses or design implications, not as empirical findings.","section":"§5.2, §6.3"}],"minor_comments":[{"comment":"There are several typos and mechanical errors that should be corrected: 'towfold' (Section 2.5), 'gesturs' (Section 2.1), 'reponse' (Section 5.3), 'knowedlge' (Section 7.1), 'In reponse' (Section 5.3), and 'compensated$15' / '$10 dollars' (Sections 3.2, 4.1).","section":"Throughout"},{"comment":"The caption of Figure 4 mentions 'externalization, socialization, and internalization' during the four stages, but the text and figure describe 'combination' as the fourth SECI process. Please align the caption with the actual coding categories.","section":"Figure 4 caption"},{"comment":"The column header 'Experience (Y ears)' contains a typo and should read 'Experience (Years)'.","section":"Table 4"},{"comment":"The reference 'EC, R.M.A., 2000' appears malformed; the author name and entry should be checked against the intended source.","section":"References"},{"comment":"The placeholder '[Anonymized for review]' should be replaced with the actual city or region in a non-anonymized version of the manuscript.","section":"§4.1"},{"comment":"The 'video demonstration via: Click Here' link is not functional in the manuscript and should either contain a real URL or be removed for review.","section":"§1 footnote"}],"recommendation":"major_revision","confidential_remarks":"The central design work is plausible and the qualitative corpus is valuable, but the current abstract and conclusions make causal claims that the single-arm design cannot bear. If the authors substantially soften the causal language and reframe the study as exploratory, the paper could become acceptable. The 'internalization' claim also needs either a real follow-up measure or removal. I would not reject outright, because the descriptive findings and design insights are likely publishable after revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Good paper to know about if you work on teachable agents or embodied professional learning. The novel bit is not the teachable agent concept, which is established, but pointing it at teachers' peer learning of instructional gestures, and doing so with a carefully built system: RAG-grounded knowledge base with cited sources, multi-agent LLM pipeline, and a skeletal mirror that genuinely seemed to reduce self-consciousness. The collaborative design phase with six teachers is a real strength, and the 30-teacher, 10-group evaluation is solid qualitative fieldwork. The vignettes in Section 5 are rich and convincingly coded with the SECI lens; they show teachers externalizing gesture knowledge through talk and movement, and co-constructing shared rationales.\n\nThe soft spots, in proportion, center on causal attribution. Section 4.2 sends all 10 groups through the same four-stage interaction, and those stages themselves require teachers to rate, comment, demonstrate, explain, and reach consensus. So the observed externalization and combination could plausibly come from the structured group activity rather than from the mentee positioning or skeletal mirror. The abstract says \"Novobo prompted\" and \"helped teachers externalize, exchange, and internalize,\" which goes beyond what the design supports. The paper acknowledges sample limits and the text-only gesture output in Section 7.6, but not this attribution gap. Also, \"internalization\" is inferred from in-session discussion and focus-group self-report; there is no post-session measure of gesture use or retention, so that part of SECI is asserted rather than shown.\n\nThese are real but not fatal. The design contribution stands on its own, and the descriptive findings are valuable. I would not want the paper rejected on these grounds, but the claims need softening or a comparison condition. This paper is for HCI and CSCL researchers interested in teacher professional development, teachable agents, generative AI in education, and embodied knowledge. It deserves a serious referee; I would send it out and ask for revisions that temper the causal language and add a limitations paragraph on attribution.","headline":"Useful design contribution and rich qualitative fieldwork, but the abstract's causal claim that Novobo prompted the learning overreaches a study design with no control condition.","tokens_in":27636,"tokens_out":1424,"would_cite":true,"duration_ms":14093,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An AI mentee, not an expert, gets teachers to externalize and share their tacit knowledge of instructional gestures.","keywords":["teachable agent","learning by teaching","instructional gestures","teacher professional development","peer learning","embodied knowledge","skeletal mirror","human-AI interaction"],"falsifier":"Conduct the same 45-minute collaborative task with the same prompts but without Novobo's interface — teachers discussing and demonstrating instructional gestures in small groups with a short script — and compare the richness of externalization and exchange. If the no-system condition produces equal or greater knowledge sharing, the central claim that Novobo's design features caused the effect fails.","tokens_in":26743,"feed_emoji":"🧑🏫","tokens_out":3770,"duration_ms":26841,"temperature":0.7,"pith_summary":"The paper introduces Novobo, a teachable AI agent that plays the role of a novice teacher seeking mentoring from a group of experienced teachers. Its central claim is that positioning the AI as a mentee, and asking teachers to evaluate, demonstrate, and explain instructional gestures, prompts teachers to externalize and share tacit embodied knowledge that they would normally keep implicit. The authors report that in ten collaborative sessions with 30 teachers, the four-step interaction cycle stimulated conversation and movement through which teachers exchanged, co-constructed, and internalized knowledge about how to gesture in class. If this is right, it would give teacher professional development a low-cost, low-pressure way to work on embodied skills that video analysis and prescriptive feedback systems handle poorly. The paper's value for HCI is the demonstration that a teachable agent, previously used almost exclusively with students, can serve as a neutral mentee that lowers the social cost of peer feedback among professionals.","feed_headline":"AI mentee prompts teachers to share and refine their gestures","feed_subtitle":"In 10 sessions with 30 teachers, the teachable agent sparked embodied knowledge exchange and co-construction.","key_machinery":"The central object is Novobo itself: a multi-agent LLM system with retrieval-augmented generation over a knowledge base of gesture types (iconic, metaphoric, deictic, emblematic), instructional intentions, and annotated classroom examples. Its interaction design is a four-step loop — posing a scenario, commenting on Novobo's suggestions, demonstrating a gesture via a skeletal mirror, and explaining it — that embodies the learning-by-teaching paradigm. The mentee role is the load-bearing social mechanism: because Novobo is positioned as a novice rather than an expert, teachers can evaluate its output without the social risk of criticizing a peer. The skeletal mirror is the load-bearing perceptual mechanism: it lets teachers see and replay their own movements without the self-consciousness of seeing their face or full appearance.","core_discovery":"The paper claims that teachers' tacit knowledge of instructional gestures can be drawn out and shared when a group collaboratively teaches an AI apprentice. Novobo generates candidate gestures for a chosen teaching scenario with referenced theory, then asks teachers to rate and comment on those suggestions, demonstrate a better gesture in front of a skeletal mirror, and explain the rationale behind it. The authors argue that this cycle maps onto the SECI knowledge-creation model: externalization happens when teachers enact and verbalize what they know, socialization happens when peers observe and imitate one another, combination happens when the group agrees on a gesture and enters a shared explanation, and internalization is the residue teachers carry back to their classrooms. The empirical claim is that the mentee positioning and the skeletal mirror were what made this flow, because teachers reported less self-consciousness and less fear of criticizing colleagues when the object of evaluation was the AI rather than a person.","pith_inferences":["The paper's design effects are not yet separated from the mere effect of teachers meeting in small groups; a control condition without Novobo, or with an expert-positioned AI, would be the natural next test.","The same mentee-agent pattern could be tested in other embodied professions, such as nursing communication or skilled manual work, where tacit knowledge is also hard to verbalize.","The system's summary of principles could be repurposed as an asynchronous medium for capturing and sharing a school's local gesture repertoire over time.","Whether the internalization claimed after one 45-minute session persists into actual classroom behavior is an open question not tested by the study."],"forward_implications":["Teacher professional development can treat embodied skills as learnable through peer exchange rather than through solitary video review or prescriptive feedback.","Teachable agents can be extended beyond student populations to support professionals whose expertise is partly tacit.","Showing referenced theoretical knowledge alongside AI suggestions increases teachers' trust in and learning from the system.","The four-stage cycle gives researchers an observable trace of externalization, socialization, combination, and internalization in embodied learning."],"supporting_citations":[{"why":"Supplies the SECI knowledge-creation model used to analyse externalization, socialization, combination, and internalization.","marker":"Nonaka, 1994"},{"why":"Provides the learning-by-teaching account that motivates positioning the agent as a mentee.","marker":"Roscoe and Chi, 2007"},{"why":"Documents teachers' reluctance to give peer criticism, the problem the mentee design is meant to solve.","marker":"Seroussi et al., 2019"},{"why":"Supplies the gesture taxonomy that grounds Novobo's knowledge base and generation of gesture types.","marker":"McNeill, 1992"},{"why":"Frames instructional gestures as tacit knowledge acquired in practice, the target of the peer-learning intervention.","marker":"Westerlund, 2021"},{"why":"Supports the claim that collaboratively teaching an agent promotes discussion and integration of different viewpoints.","marker":"Yu et al., 2009"},{"why":"Represents the prior AI-based gesture training approach the authors position against as prescriptive and one-size-fits-all.","marker":"Barmaki and Hughes, 2018"}],"fun_headline_variants":["AI mentee prompts teachers to share gesture know-how","Teaching an AI apprentice makes teachers reflect on gestures","Mentee AI spurs collaborative gesture learning in teachers","Teachers externalize tacit gesture knowledge by mentoring AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The study assumes that the knowledge exchange it observed came from Novobo's mentee positioning and skeletal mirror rather than from the simple fact that teachers sat together in small groups and talked about gestures, since every group used Novobo and there was no comparison condition.","fun_headline_variants_meta":{"raw":{"variants":["AI mentee prompts teachers to share gesture know-how","Teaching an AI apprentice makes teachers reflect on gestures","Mentee AI spurs collaborative gesture learning in teachers","Teachers externalize tacit gesture knowledge by mentoring AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000207,"raw_usage":{"total_tokens":1390,"prompt_tokens":924,"completion_tokens":466,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":404}},"tokens_in":540,"tokens_out":466,"duration_ms":4300,"temperature":1.0,"reasoning_tokens":404,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:43:37.327828+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Conduct the same 45-minute collaborative task with the same prompts but without Novobo's interface — teachers discussing and demonstrating instructional gestures in small groups with a short script — and compare the richness of externalization and exchange. If the no-system condition produces equal or greater knowledge sharing, the central claim that Novobo's design features caused the effect fails.","supporting_citations":[{"cited_title":", year 1994","cited_arxiv_id":null,"evidence_quote":"Supplies the SECI knowledge-creation model used to analyse externalization, socialization, combination, and internalization."},{"cited_title":", author Chi, M.T","cited_arxiv_id":null,"evidence_quote":"Provides the learning-by-teaching account that motivates positioning the agent as a mentee."},{"cited_title":", author Sharon, R","cited_arxiv_id":null,"evidence_quote":"Documents teachers' reluctance to give peer criticism, the problem the mentee design is meant to solve."},{"cited_title":", year 1992","cited_arxiv_id":null,"evidence_quote":"Supplies the gesture taxonomy that grounds Novobo's knowledge base and generation of gesture types."},{"cited_title":", year 2021","cited_arxiv_id":null,"evidence_quote":"Frames instructional gestures as tacit knowledge acquired in practice, the target of the peer-learning intervention."}],"review_version":1}