{"id":"3d2d8a18-9436-4b8e-9502-2d14a8f36306","arxiv_id":"2605.30539","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Copa is a theory-guided multimodal LLM agent that supports high school computational modeling through adaptive feedback, shown in a 33-dyad study to increase student confidence and conceptual verbalization without fostering dependence.","lead":"The paper introduces Copa, a multi-agent LLM pedagogical system for high school STEM+C learning that uses an Evidence-Decision-Feedback framework grounded in Social Cognitive Theory and Social Constructivism to provide adaptive dialogic support. A smart generalist might read it to see one concrete attempt at designing classroom AI that builds student reasoning instead of replacing it.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"The claim that EDF grounding prevents over-reliance rests on unverified measurement details from the n=33 dyad study.","rationale":"The reader's weakest_assumption directly identifies the measurement gap that blocks verification of the strongest_claim. No additional internal inconsistency appears from the abstract; the UNVERDICTED status holds until full methods and data are examined.","tokens_in":1692,"tokens_out":255,"duration_ms":14811,"concrete_test":"Extract the methods and results sections describing the dependence measures and any control conditions; recompute or inspect whether independent problem-solving scores differ significantly between Copa and baseline conditions, or whether reliance metrics are limited to self-report.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the theory-guided EDF interactions demonstrably avoid cognitive offloading, yet the abstract provides no specifics on how dependence was operationalized or measured (e.g., independent post-test accuracy, usage frequency without prompts, or comparison to a no-agent control). The assumption that Social Cognitive Theory + Social Constructivism via EDF is sufficient therefore remains the least secure link; without those metrics or controls, the study cannot isolate the framework's effect from confounds such as dyad collaboration or multimodal novelty.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces Copa, an agentic multi-agent multimodal Collaborative Peer Agent for STEM+C learning built on the Evidence-Decision-Feedback (EDF) framework, which grounds interactions in Social Cognitive Theory and Social Constructivism to promote sense-making through adaptive dialogic support. Based on an authentic high school computational-modeling study with n=33 dyads, the paper claims that Copa supports students' confidence building and ability to verbalize conceptual understanding without causing dependence, and provides adaptive feedback personalized to learners that is interpretable with respect to students' multimodal input data.","tokens_in":1801,"tokens_out":450,"duration_ms":23696,"significance":"If the reported findings from the user study hold under rigorous scrutiny, this work could be significant in advancing the integration of LLM-based pedagogical agents in education. By explicitly grounding the agent in established learning theories via the EDF framework, it offers a potential solution to concerns about cognitive offloading and over-reliance, positioning theory-guided multimodal agents as a viable approach for amplifying student reasoning in classroom settings.","major_comments":[{"comment":"The abstract asserts empirical results from the 33-dyad study demonstrating that Copa supports confidence without dependence, but supplies no data, statistical methods, error bars, baseline comparisons, or exclusion criteria. This makes it impossible to determine whether the data support the claims as stated regarding the prevention of over-reliance.","section":"Abstract"},{"comment":"The claim that the EDF framework prevents over-reliance is load-bearing for the central contribution, yet the manuscript provides no specifics on how dependence was operationalized or measured (e.g., independent post-test accuracy, usage frequency without prompts, or comparison to a no-agent control), leaving the isolation of the framework's effect from potential confounds unaddressed.","section":"User Study Description"}],"minor_comments":[{"comment":"The abstract could benefit from a brief mention of the specific STEM+C topic or computational modeling task used in the study to provide context for the claims.","section":null}],"recommendation":"major_revision","confidential_remarks":"The lack of empirical details in the abstract raises concerns about whether the full manuscript adequately presents the study methodology and results for reproducibility and assessment."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive comments, which highlight opportunities to strengthen the clarity and transparency of our empirical claims. We address each major point below and indicate where revisions will be made.","responses":[{"response":"We agree that the abstract, due to length constraints, presents high-level claims without the supporting statistical details. The full manuscript contains these elements in the Results section (including pre/post confidence measures, verbalization coding, dependence proxies via post-test accuracy and interaction logs, and baseline comparisons). To improve accessibility, we will revise the abstract to briefly note the study design (within-subjects dyad comparison with multimodal logging) and key statistical outcomes while preserving conciseness.","revision_made":"yes","referee_comment":"[Abstract] The abstract asserts empirical results from the 33-dyad study demonstrating that Copa supports confidence without dependence, but supplies no data, statistical methods, error bars, baseline comparisons, or exclusion criteria. This makes it impossible to determine whether the data support the claims as stated regarding the prevention of over-reliance."},{"response":"The referee correctly identifies that the current manuscript text does not explicitly detail the operationalization of dependence in the User Study Description. Dependence was assessed via (1) independent post-test accuracy on modeling tasks without agent access, (2) frequency of agent queries versus self-initiated actions in logs, and (3) comparison against a no-agent control condition within the dyad design. We will expand this section with these metrics, exclusion criteria, and statistical methods to allow readers to evaluate isolation of the EDF effect from confounds.","revision_made":"yes","referee_comment":"[User Study Description] The claim that the EDF framework prevents over-reliance is load-bearing for the central contribution, yet the manuscript provides no specifics on how dependence was operationalized or measured (e.g., independent post-test accuracy, usage frequency without prompts, or comparison to a no-agent control), leaving the isolation of the framework's effect from potential confounds unaddressed."}],"tokens_in":1360,"tokens_out":401,"duration_ms":14211,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's main new piece is the Copa agent itself: a multi-agent, multimodal setup that routes interactions through an Evidence-Decision-Feedback loop explicitly drawn from Social Cognitive Theory and Social Constructivism. The authors position this as a way to promote sense-making and adaptive feedback instead of direct answers. They also report outcomes from an authentic high school computational modeling study with 33 dyads. That specific implementation and the reported classroom results are not in the prior work they cite.\n\nThe design choice to ground the agent in those two theories is a clear strength. It moves past generic prompting and tries to make the agent's behavior interpretable with respect to students' multimodal inputs. The emphasis on verbalization and confidence building without dependence is the right target.\n\nThe soft spot is the study evidence. The abstract states that Copa supports confidence and verbalization without causing dependence and that the feedback is adaptive and personalized, yet it supplies no operational definitions, no description of how dependence was measured, no baseline comparisons, and no statistical results. The stress-test note is correct on this point: without those details it is not possible to isolate the effect of the EDF framework from dyad collaboration or other factors. The central assumption that theory grounding alone is enough to prevent cognitive offloading therefore rests on unshown measurement choices.\n\nIf the full manuscript contains the methods, data, and analysis that are missing from the abstract, the work could be useful to people building educational agents who want a concrete example of theory-driven scaffolding. Readers working on LLM agents for classrooms would get the most from it. The paper shows honest engagement with the over-reliance literature even if the current presentation of results is incomplete.\n\nI would send it to peer review. The topic matters and the framework is a reasonable starting point, but referees will need to see the actual study design and numbers before the claims can be evaluated.","headline":"Copa applies learning theory via an EDF framework to an LLM agent for STEM+C and reports a 33-dyad study claiming reduced over-reliance, but the abstract gives no measurement details or stats to support that claim.","tokens_in":2338,"tokens_out":465,"would_cite":false,"duration_ms":20169,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A theory-grounded multimodal LLM agent supports student confidence in STEM+C without causing dependence.","keywords":["LLM pedagogical agent","STEM+C scaffolding","over-reliance","cognitive offloading","Evidence-Decision-Feedback","Social Cognitive Theory","multimodal agent","high school study"],"falsifier":"A replication study finding that students using Copa show increased dependence on the agent or reduced ability to solve problems independently compared to a control group.","tokens_in":2604,"feed_emoji":"🤖","tokens_out":593,"duration_ms":21671,"temperature":0.7,"pith_summary":"The paper develops Copa, a multi-agent multimodal LLM pedagogical agent for STEM+C learning. Copa uses the Evidence-Decision-Feedback framework to ground its actions in Social Cognitive Theory and Social Constructivism, aiming for adaptive dialogic support that promotes sense-making. A study with 33 high school dyads shows the agent helps students build confidence and verbalize understanding without leading to over-reliance or dependence. This addresses concerns that LLM agents in education might encourage cognitive offloading. The findings suggest theory-guided design can make classroom AI integration more beneficial by amplifying student reasoning.","feed_headline":"Theory-guided LLM agent builds student confidence without dependence","feed_subtitle":"High school study of 33 dyads shows adaptive feedback helps verbalize concepts while avoiding over-reliance on the AI.","key_machinery":"The Evidence-Decision-Feedback (EDF) framework that structures the agent's interactions to promote sense-making through adaptive, dialogic support rather than answer-seeking.","core_discovery":"Copa demonstrates that an agentic, multi-agent, multimodal Collaborative Peer Agent, built on the Evidence-Decision-Feedback framework, can support students' confidence building and ability to verbalize conceptual understanding without causing dependence while providing adaptive feedback personalized to learners and interpretable with respect to their multimodal input data.","pith_inferences":["The EDF approach could be adapted for other AI tutoring systems to reduce over-reliance risks.","Testing the agent in different subject areas might reveal its broader applicability.","Long-term studies could check if students maintain independent problem-solving skills after using the agent.","Combining the framework with additional data sources like eye-tracking could refine personalization further."],"forward_implications":["Students build confidence in their STEM+C abilities through guided interactions.","Learners improve their ability to verbalize conceptual understanding.","The agent delivers adaptive feedback based on students' multimodal inputs without fostering dependence.","Such theory-guided agents offer a path for AI in classrooms that enhances rather than replaces student reasoning."],"fun_headline_variants":["Theory-guided Copa fosters confidence without learner dependence","Multimodal agent supports verbalizing concepts independently","EDF-based LLM agent promotes sense-making in STEM+C","Adaptive feedback from theory-driven agent avoids over-reliance"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That interactions grounded in the learning theories via the EDF framework sufficiently prevent cognitive offloading and over-reliance as measured in the study.","fun_headline_variants_meta":{"raw":{"variants":["Theory-guided Copa fosters confidence without learner dependence","Multimodal agent supports verbalizing concepts independently","EDF-based LLM agent promotes sense-making in STEM+C","Adaptive feedback from theory-driven agent avoids over-reliance"]},"model":"grok-4.3","cost_usd":0.003529,"raw_usage":{"total_tokens":1830,"prompt_tokens":622,"num_sources_used":0,"completion_tokens":57,"cost_in_usd_ticks":35287000,"prompt_tokens_details":{"text_tokens":622,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1151,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":622,"tokens_out":57,"duration_ms":8686,"temperature":1.0,"reasoning_tokens":1151,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T23:41:30.051096+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A replication study finding that students using Copa show increased dependence on the agent or reduced ability to solve problems independently compared to a control group.","supporting_citations":[],"review_version":1}