{"id":"9521691e-25c7-41c4-9b81-0650a42b81df","arxiv_id":"2512.18239","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Hidden contrarian AI teammates increased challenge and reflection in group chat but reduced psychological safety and satisfaction, with no gain in creative performance.","lead":"An experiment with 97 online triads shows that an undisclosed AI teammate that argues and challenges changes how groups talk—more critical back-and-forth—but lowers people's satisfaction and psychological safety without improving their creative output. A supportive hidden AI instead keeps conversations agreeable and smooth.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pooled AI+human discourse analyses cannot support the 'reshaped learner agency' claim; human-only reanalysis is needed.","rationale":"The reader's conditional verdict is well-founded. The affective findings—contrarian AI reducing satisfaction and safety without creative gains—are supported by human-only regressions and are the most robust contribution. The process-level claims, however, are the crux of the title and RQs, and they are undermined by pooling. The authors could easily address this with a human-only reanalysis; if the patterns vanish, the 'learner agency' framing should be dropped or softened to 'hybrid team discourse.' My concern does not change the verdict level, because the reader already identified this as the weakest assumption and assigned CONDITIONAL. I would not accept as-is without this check.","tokens_in":24562,"tokens_out":5233,"duration_ms":56846,"concrete_test":"Re-run the RQ1 permutation tests (Section 3.6) and RQ2 motif Fisher tests (Section 3.7) on the same coded sequences after deleting all AI utterances, preserving chronological order of human messages. If the significant condition differences in transition probabilities and motif odds ratios largely disappear, the discourse restructuring is attributable to the AI's scripted output rather than to reshaped learner agency; if they persist, the 'regulatory attractor' interpretation gains support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing inference is in RQ1–RQ3 (Sections 3.6–3.8), where all transition networks, sequential motifs, and GMM profiles are computed on sequences that pool human and AI utterances. Section 3.6 explicitly states that 'All utterances from humans and AI were included.' The observed contrasts—contrarian conditions showing stronger transitions into Challenge, Productive Friction and Challenge-Integration motifs, supportive conditions showing Agreement-centred pathways—are exactly what one would expect if the AI simply followed its scripted persona: the contrarian agent was instructed to challenge and dismiss suggestions, the supportive agent to affirm and invite input (Appendix B). The paper does not report human-only transition matrices, human-only motif prevalences, or a test of whether human speakers' cluster memberships differ across conditions. Section 4.3 even shows AI agents concentrating in a Hard Challenger cluster and 'reflective regulation uniquely human,' which is consistent with the persona scripts rather than with emergent human adaptation. Section 5.1's interpretation of personas as 'regulatory attractors that reweight what becomes easy, likely, and legitimate to do next' therefore rests on an unstated assumption: that the pooled differences reflect changes in human behaviour. The affective outcomes (RQ4) are human-only and robust, but they do not support the structural/temporal/role-based 'learner agency' claim. This is not an outside-consensus dispute; it is an internal mismatch between the evidence and the construct.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a randomized online experiment (N=224, 97 triads) comparing human-only, supportive-AI, and contrarian-AI teams on a collaborative movie-plot task. AI teammates were undisclosed. The authors code discourse into seven creative-regulatory categories and use transition network analysis (TNA), sequential pattern mining (SPM), and Gaussian mixture models (GMM) to characterize transition networks, sequential motifs, and speaker profiles. They link these to post-task questionnaires and a creativity-change score. Main findings: contrarian AI yields more challenge- and reflection-rich discourse and motifs, supportive AI yields agreement-centred trajectories; AI agents concentrate in challenger profiles and reflective regulation is uniquely human; contrarian AI reduces teamwork satisfaction and psychological safety, with no effects on cognitive load or creative performance.","tokens_in":24895,"tokens_out":4341,"duration_ms":42031,"significance":"If supported, the paper would make a valuable contribution by showing that persona-driven AI can systematically restructure collaborative discourse even when participants are unaware of AI presence. The randomized design, validated coding (κ = .94), implicit-AI manipulation with a low detection check, and the robustness of the affective outcome regressions are strengths. However, the discourse-level evidence for 'reshaped learner agency' is undermined by pooling AI and human utterances, so the significance depends on a reanalysis.","major_comments":[{"comment":"The load-bearing claim of RQ1–RQ3 is that personas 'reweight what becomes easy, likely, and legitimate to do next' (§5.1) and thereby reshape learner agency. However, all transition networks, sequential motifs, and cluster features are computed on pooled human+AI utterances: §3.6 states 'All utterances from humans and AI were included'; §3.8 aggregates 'all coded utterances from both human and AI speakers.' Since the persona prompts (Appendix B) explicitly instruct the contrarian agent to 'challenge or dismiss suggestions' and the supportive agent to 'affirm... invite input,' the observed condition contrasts are exactly what the scripted AI would produce by itself. No human-only transition matrices, human-only motif prevalences, or tests of human cluster-membership differences across conditions are reported. The conclusion that AI personas reshaped human agency therefore conflates the AI","section":"§3.6–3.8, §4.1–4.3, §5.1"},{"comment":"TNA permutes 'every network feature' without any described correction for multiple comparisons. The number of tested edges and centrality measures is large, and several reported contrasts are modest (e.g., p = .022, .037, .039). These could disappear under FDR or family-wise control. Please report the total number of tests, the correction procedure, and adjusted p-values, or justify why correction is unnecessary.","section":"§3.6, §4.1"},{"comment":"The GMM solution has silhouette = 0.035, indicating essentially no separation, and several clusters are extremely small (n = 7, n = 5, n = 8). The assertions that AI agents concentrate in 'Hard Challenger' and that 'Reflective Regulator' is 'uniquely human' rest on tiny, poorly separated groups. Given that AI features are determined by the persona prompts, the profile asymmetry is also by construction. I recommend reporting cluster stability (e.g., bootstrap resampling) and, at minimum, the human-only cluster distributions across conditions.","section":"§3.8, §4.3"},{"comment":"The manuscript states that Fisher exact tests for motifs were corrected using Holm-Bonferroni, but §4.2 reports raw p-values without indicating adjusted values. For Productive Friction the reported p = .045 and p = .023 would likely not survive correction across five motifs and three pairwise comparisons; Reflective Cycle is non-significant. The motif-level claims should be based on adjusted p-values and confidence intervals.","section":"§3.7, §4.2"}],"minor_comments":[{"comment":"The keywords list 'Moral reasoning; Moral foundations', which appear unrelated to the manuscript. These should be removed or replaced with relevant terms (e.g., AI persona, collaborative creativity, learner agency).","section":"Title page (keywords)"},{"comment":"The typo 'RE.S.PONSE' appears multiple times in both persona prompts. Correct to 'RESPONSE'.","section":"Appendix B"},{"comment":"The fourth motif is labeled 'Challenge Integration' and described as Challenge→Elaboration→Integration, but §4.2 refers to an 'Elaboration Agreement' motif as Elaboration→Agreement→Integration. Labels should be consistent and match the table.","section":"Table 3 / §4.2"},{"comment":"Motif prevalences are reported as percentages with odds ratios relative to the Contrarian condition (e.g., Control OR = 0.25). Clarify the reference category in the text and figure to avoid reader confusion about the direction of effects.","section":"§4.2"},{"comment":"The AI model is described as 'GPT-5-based' but no version, temperature, or API parameters are given. Provide model version and key configuration parameters for reproducibility.","section":"§3.3"},{"comment":"In §2.5 the citation 'Council, 2012' is used, but the reference list entries use 'N. R. Council' and 'Council, N. R.' — standardize the citation name.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The pooled human+AI discourse analysis is the core technical issue. If the authors can provide human-only reanalyses showing condition differences in human transitions, motifs, or cluster memberships, the paper could be strong. The affective outcome findings are solid and should not be buried in the revision. I would also encourage the editor to consider the manuscript's fit with the journal's scope given the keywords issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The affective finding is the real contribution here: in an undisclosed-AI collaborative creativity task, a contrarian AI teammate reduced teamwork satisfaction and psychological safety without improving creative output. That result is clean, human-only, and practically important. The experiment itself is well built: randomized triads, a control condition, validated scales, low AI detection rates, and careful coding with high reliability. This is genuinely new relative to the explicit-AI literature.\n\nThe soft spot is the process-level claim. Sections 3.6, 3.7, and 3.8 all pool AI and human utterances. The contrarian AI was explicitly prompted to challenge and dismiss suggestions, so the stronger transitions into Challenge, the Productive Friction and Challenge-Integration motifs, and the all-AI Hard Challenger cluster are exactly what the script tells the AI to do. That does not tell us that learners' own behavior changed. The paper never reports human-only transition networks, human-only motif prevalence, or a test of whether human cluster memberships differ across conditions. The title says \"Emergent Learner Agency,\" but agency is about the learners, and the evidence as presented is about the whole team's discourse, which includes the AI's scripted moves. I don't see how the central inference can survive without a human-only reanalysis.\n\nThere are smaller issues that reinforce the caution: the TNA permutation tests don't appear to correct for multiple comparisons across many edges; the motif p-values are borderline even with Holm–Bonferroni; the GMM silhouette is 0.035, which is weak separation; and the code and data aren't public. None of these are fatal on their own, but they compress the room for confidence.\n\nWho should read this? People designing AI teammates for education or CSCL, and anyone who works on human-AI interaction methods. It is also a good teaching example of why you must separate the manipulation from the outcome measure when the \"teammate\" is part of the measured process.\n\nMy recommendation: this deserves peer review, not a desk rejection. The affective result is publishable, and the experiment is more careful than most. But a serious referee should require human-only reanalyses (or at least a decomposition showing human behavior shifts) before accepting the \"reshaped learner agency\" framing. If the authors can show that human transition patterns change in response to the AI, the paper becomes much stronger. Right now, the title overstates what the data can support.","headline":"Solid experiment, shaky inference: the affective results hold up, but the discourse analyses can't separate what the AI did from what the learners actually did.","tokens_in":25361,"tokens_out":3661,"would_cite":false,"duration_ms":41598,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AI teammates with scripted personas reshape how small groups talk and feel, even when no one knows the AI is there, while leaving creative output unchanged.","keywords":["human-AI collaboration","learner agency","AI persona","implicit AI","collaborative creativity","psychological safety","transition network analysis","sequential pattern mining"],"falsifier":"Re-analyze the chat logs with all AI utterances removed, recomputing transition networks, motif frequencies, and cluster membership for humans only. If the Contrarian and Supportive conditions no longer differ from each other or from Control on these human-only measures, the claim that AI personas reshape learner agency is falsified.","tokens_in":24493,"feed_emoji":"🤖","tokens_out":3360,"duration_ms":34436,"temperature":0.7,"pith_summary":"This paper claims that an AI teammate with a scripted persona can change the course of a small group's creative work without anyone knowing the AI is present. In a randomized experiment, 224 students in 97 triads wrote movie plots in text chat with either two humans, two humans plus a supportive AI, or two humans plus a contrarian AI. Discourse coding and network/sequence analyses showed the contrarian persona pulled conversation toward challenge and reflection, while the supportive persona produced agreement-centred convergence. The same contrarian condition lowered teamwork satisfaction and psychological safety and did not improve creative performance. The paper argues this reveals a design tension: cognitive friction can be created artificially, but it carries affective costs that do not buy better output.","feed_headline":"Contrarian AI teammates cut psychological safety, not creativity","feed_subtitle":"Even when learners never spot the bot, its persona rewires group talk: challenge and reflection rise, satisfaction and safety drop.","key_machinery":"The paper's central instrument is a seven-code creative-regulatory framework (Idea, Elaboration, Challenge, Agreement, Integration, Reflection, Off-task) applied to every chat utterance; transitions among these codes are modeled as weighted directed networks (transition network analysis), recurring multi-step sequences are mined and checked against five theory-defined motifs such as Productive Friction (Idea→Challenge→Integration) and Safe Convergence (Idea→Agreement→Integration), and speaker-level code proportions are clustered with Gaussian mixture models. This combination lets the authors read agency as structure (who connects to what), time (which sequences recur), and role (which profil","core_discovery":"Persona-driven AI operating invisibly acts as a regulatory attractor: it reweights which conversational moves become likely next steps, creating distinct structural and temporal pathways. Contrarian AI produced substantially stronger transitions into Challenge from nearly every other state, sustained Challenge persistence, and stronger pathways into Reflection, together with a Challenge→Elaboration→Integration motif that appeared only in contrarian teams. Supportive AI stabilized Idea and Agreement trajectories, including a dominant Idea→Agreement→Integration motif. Clustering showed AI agents concentrated in challenger-oriented profiles, while the reflective-regulator profile contained only","pith_inferences":["Because the reported transitions and motifs pool AI and human utterances, the cleanest test of 'reshaped learner agency' would be a human-only reanalysis with AI messages removed; this is not reported in the paper, so the agency interpretation remains an inference.","If personas are governance knobs, an adaptive AI that starts supportive and introduces bounded challenge later may preserve safety while still getting critical engagement; that is directly testable in a follow-up experiment.","The absence of creative gains may reflect the 10-minute task; longer or multi-session collaboration could reveal whether friction converts to better output or whether the affective cost compounds over time.","The uniquely human reflective-regulator cluster suggests a useful division of labor: AI can supply dissent, but meta-level monitoring may remain a human function, and future agents that prompt reflection rather than perform it might better protect learner agency."],"forward_implications":["If an AI participates without disclosure, its persona can steer discussion regardless of learners' awareness, so 'invisible' AI is still an active governance force in collaboration.","Contrarian personas can institutionalize critique and reflection as routine discourse, but in a single short session this does not translate into measurable creative gains.","Affective climate is driven more by the overall persona tone than by individual discourse profiles: psychological safety fell in contrarian teams even though no specific agency cluster accounted for the drop.","Supportive personas can keep ideation flowing and build cohesion, but they may reduce critical engagement and did not raise psychological safety beyond the human-only baseline.","Educational designs should treat challenge as bounded friction: introduce it late or sparingly, pair it with integrative prompts, and add repair moves to protect the affective climate."],"fun_headline_variants":["Hidden contrarian AI bolsters debate but dampens team trust","Contrarian AI spurs pushback but not better ideas","Invisible AI teammate boosts challenge, slashes team safety","AI persona trade-off: more debate, less safety, same creativity"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the measured differences in transition networks, motifs, and clusters represent a change in learners' emergent agency, even though AI and human utterances are pooled; if those differences mostly reflect the AI's own scripted messages rather than shifts in human behavior, the central interpretation collapses.","fun_headline_variants_meta":{"raw":{"variants":["Hidden contrarian AI bolsters debate but dampens team trust","Contrarian AI spurs pushback but not better ideas","Invisible AI teammate boosts challenge, slashes team safety","AI persona trade-off: more debate, less safety, same creativity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001058,"raw_usage":{"total_tokens":4297,"prompt_tokens":784,"completion_tokens":3513,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":3442}},"tokens_in":528,"tokens_out":3513,"duration_ms":23959,"temperature":1.0,"reasoning_tokens":3442,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T15:02:42.137973+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-analyze the chat logs with all AI utterances removed, recomputing transition networks, motif frequencies, and cluster membership for humans only. If the Contrarian and Supportive conditions no longer differ from each other or from Control on these human-only measures, the claim that AI personas reshape learner agency is falsified.","supporting_citations":[],"review_version":1}