pith. sign in

arxiv: 2606.27938 · v2 · pith:TPVU7S6Onew · submitted 2026-06-26 · ⚛️ physics.soc-ph

Students using GenAI lag behind in problem-solving competence: an agent-based study of classroom networks

Pith reviewed 2026-06-30 10:11 UTC · model grok-4.3

classification ⚛️ physics.soc-ph
keywords generative AIproblem-solving competenceagent-based modelclassroom networkscompetence developmentpeer collaborationhigh school physicseducational simulation
0
0 comments X

The pith

Generative AI access in high school classrooms reduces problem-solving competence development and leaves more students in lower tiers.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds an agent-based simulation of high school physics students developing problem-solving competence through individual work, peer collaboration, or GenAI assistance. It runs the model on classrooms with and without GenAI across varied peer-network structures to compare outcomes. The central finding is that GenAI access produces slower competence gains overall and raises the fraction of students who stay in lower competence categories. A sympathetic reader would care because the result points to collective effects on classroom learning dynamics that go beyond any single student's use of the tool.

Core claim

Simulations of classroom networks show that adding GenAI support for tasks diminishes the growth of problem-solving competence and increases the share of students who remain in lower competence tiers, with the effect appearing across different peer-interaction structures.

What carries the argument

An agent-based model in which students update competence through individual tasks, peer collaboration, or GenAI offloading, with network topology modulating the interactions.

If this is right

  • Classrooms using GenAI exhibit slower average competence growth than matched classrooms without it.
  • The fraction of students remaining in lower competence tiers rises when GenAI is available.
  • Varying the structure of peer networks does not remove the reduction in competence development caused by GenAI.
  • Evaluating GenAI in education requires tracking collective competence dynamics, not only individual outcomes.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Task designs that limit easy offloading to GenAI might be needed to preserve the competence-building role of peer work.
  • The same pattern could appear in subjects other than physics if the underlying offloading and collaboration mechanisms hold.
  • Schools considering broad GenAI rollout may need to monitor tier distributions rather than only average test scores.
  • Longer simulations could test whether the competence gap widens or stabilizes over multiple semesters.

Load-bearing premise

The model's rules for how GenAI offloads cognitive effort, how peer work builds competence, and how network structure changes these processes match real high school classroom behavior.

What would settle it

Longitudinal measurement of problem-solving competence scores in real high school classes that do and do not allow GenAI, checking whether the groups diverge in the predicted direction on both average growth and the size of the lowest tier.

Figures

Figures reproduced from arXiv: 2606.27938 by Carsten K\"allner, Chenyu Li, Giulia Lorenzini, Iacopo Caporossi, Ilaria Stanzani, Karolina Levanait\.e, Lorenzo Betti, Marta Baratto, Michele Re Fiorentin, Vittoria Socci, Xuan-Chen Liu.

Figure 1
Figure 1. Figure 1: Performance comparison between passing student and GenAI scores: (A) Mean scores by exercise group between students (blue), that pass the exercise, and GenAI responses (orange). (B) Breakdown of the scoring difference between student and GenAI responses by exercise group and skills required (i.e., Maths, Physics, problem-solving , Total.) We use different data sources to both initialise our model and infor… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the agents’ dynamics and characteristics. The diagram represents the learning process of each agent (representing a student of the classroom), connected to its peers through a network and engaged in solving physics exercises. At each time step t the agent (1) choose to use or not use GenAI, (2) interact with their peers, (3) solve an exams, (4) update their problem-solving competence and GenAI … view at source ↗
Figure 3
Figure 3. Figure 3: Classroom state at two consecutive time steps. Yellow agents are students using GenAI, while red and blue agents are students not using GenAI, respectively cooperating with peers and studying alone. Dashed links represent the underlying network, while red links indicate active peer collaborations. At each time step in the dynamics, the agents first choose their strategies s t i , deciding whether to use Ge… view at source ↗
Figure 4
Figure 4. Figure 4: Student Competency Trajectories: Plots showing the trajectories of the mean student competency whether GenAI is present in the classrooms (blue) or not present (orange) as strategy for students by all four network types. no access to GenAI leads to a relatively narrow final distribution concentrated in the upper competence tiers, having access to GenAI maintains a larger lower-competence segment of the cla… view at source ↗
Figure 5
Figure 5. Figure 5: Student mobility across PSC tiers. Sankey diagrams showing transitions between five PSC tiers at t = 0, 30, 60 in the no-GenAI (A) and GenAI (B) conditions. The tiers are obtained by splitting the PSC range into five equally spaced intervals: Very Low, Low, Mid, High, and Very High. This refers to the modified SBM interaction network. 3.3 GenAI access splits the classroom into two competence groups The fac… view at source ↗
Figure 6
Figure 6. Figure 6: Competence splits by use: Competence histograms at t = 1, 20, 40, 60 for (A) the Watts-Strogatz network and (B) the modified stochastic block model, with students grouped by GenAI-usage frequency (0-100%). A single Gaussian-like peak separates into two over time, with the heaviest GenAI users massing in the low-competence peak with heavy GenAI reliance suppressing mastery and drives stratification. This be… view at source ↗
read the original abstract

The development of problem-solving competence (PSC) among high school students is foundational for preparing resilient and adaptive citizens. Generative artificial intelligence (GenAI) can support this process, but it may also encourage students to offload part of the cognitive work that is necessary for deep learning. While the individual effects of GenAI use are increasingly studied, its collective consequences for competence development within classroom environments remain underexplored. In this study, we use an agent-based model to simulate the evolution of PSC in a high school physics classroom, where students complete tasks individually, in collaboration with peers, or with the support of GenAI. By comparing classrooms with and without access to GenAI across different peer-network structures, we show that GenAI use can diminish competence development and increase the share of students remaining in lower competence tiers. These results suggest that the educational impact of GenAI should be assessed not only through individual learning outcomes but also through its effects on collective competence dynamics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper claims that an agent-based model simulating problem-solving competence (PSC) evolution in high school physics classrooms demonstrates that GenAI access diminishes competence development and increases the share of students remaining in lower competence tiers, with outcomes depending on peer-network structures.

Significance. If the simulation outcomes prove robust, the work draws attention to collective, network-mediated effects of GenAI on classroom learning that go beyond individual-level studies, offering a computational lens for assessing educational technology impacts.

major comments (2)
  1. [Model specification] Model specification section: the functional forms governing GenAI offloading of cognitive work and peer-edge competence transfer are not derived from or fitted to classroom data; the headline result that GenAI increases the fraction of students in lower PSC tiers is produced by these specific choices, and the skeptic note indicates that plausible alternative rules can reverse the sign of the effect.
  2. [Results and validation] Results and validation sections: no sensitivity analysis on the offloading or transfer rules and no external validation against real high-school classroom data are reported, which is load-bearing for the comparative claim across GenAI conditions and network topologies.
minor comments (1)
  1. [Abstract] Abstract: the specific network topologies examined and the quantitative definitions of competence tiers could be stated more explicitly to allow readers to assess the scope of the reported effects.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the detailed and constructive comments. We address the two major points below, clarifying the scope of our simulation study while committing to improvements where feasible.

read point-by-point responses
  1. Referee: [Model specification] Model specification section: the functional forms governing GenAI offloading of cognitive work and peer-edge competence transfer are not derived from or fitted to classroom data; the headline result that GenAI increases the fraction of students in lower PSC tiers is produced by these specific choices, and the skeptic note indicates that plausible alternative rules can reverse the sign of the effect.

    Authors: We agree that the functional forms are theoretical assumptions drawn from the cognitive offloading and social learning literatures rather than statistically fitted to classroom datasets. This is inherent to the exploratory agent-based modeling approach, which seeks to illustrate possible collective dynamics under stated mechanisms rather than to predict empirical outcomes. The skeptic note in the manuscript already flags that alternative rules could reverse the sign, consistent with the referee's observation. We will expand the methods section to more explicitly justify each functional form from prior theory and will add a dedicated sensitivity analysis subsection in the revision. revision: partial

  2. Referee: [Results and validation] Results and validation sections: no sensitivity analysis on the offloading or transfer rules and no external validation against real high-school classroom data are reported, which is load-bearing for the comparative claim across GenAI conditions and network topologies.

    Authors: We acknowledge that sensitivity analysis on the offloading and transfer parameters was not reported and will add it to the revised results section, including systematic variation of the key exponents and thresholds to test robustness of the tier-distribution outcomes. External validation against real high-school data is not feasible within the current purely simulation-based study; the model is intended as a computational lens for hypothesis generation rather than calibrated prediction. revision: partial

standing simulated objections not resolved
  • External validation against real high-school classroom data

Circularity Check

0 steps flagged

No circularity: simulation outputs follow directly from stated model rules without self-referential reduction

full rationale

The paper describes an agent-based simulation comparing GenAI and no-GenAI conditions across network structures. No equations, fitted parameters, or derivation chain are presented that would make any reported outcome equivalent to its inputs by construction. The functional forms for competence growth, offloading, and peer transfer are model assumptions whose consequences are then computed; this is standard simulation practice and does not constitute self-definition, fitted-input prediction, or load-bearing self-citation. The comparative claim is therefore an output of the chosen rules rather than a tautology. No steps meet the criteria for circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Abstract-only review; full model specification unavailable. The claim rests on unstated simulation rules for competence evolution and GenAI interaction effects.

axioms (1)
  • domain assumption Agent-based models with the chosen interaction rules can faithfully represent real student competence development with and without GenAI
    Invoked implicitly to extrapolate from simulation to educational implications.

pith-pipeline@v0.9.1-grok · 5750 in / 1071 out tokens · 35457 ms · 2026-06-30T10:11:30.171870+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

60 extracted references · 30 canonical work pages · 1 internal anchor

  1. [1]

    Pedagogical Applications of Generative AI in Higher Education: A Systematic Review of the Field , url =

    Qian, Yufeng , date =. Pedagogical Applications of Generative AI in Higher Education: A Systematic Review of the Field , url =. TechTrends , number =. 2025 , bdsk-url-1 =. doi:10.1007/s11528-025-01100-1 , id =

  2. [2]

    2023 , publisher =

    Guidance for Generative AI in Education and Research , author =. 2023 , publisher =. doi:10.54675/EWZM9535 , isbn =

  3. [3]

    A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education , journal=

    L. A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education , journal=. 2025 , month=. doi:10.1038/s41539-025-00320-7 , url=

  4. [4]

    Journal of Computer Assisted Learning , volume =

    Wu, Ting-Ting and Lee, Hsin-Yu and Chen, Pei-Hua and Lin, Chia-Ju and Huang, Yueh-Min , title =. Journal of Computer Assisted Learning , volume =. doi:https://doi.org/10.1111/jcal.13085 , year =

  5. [5]

    Investigating peer influence on collaborative group members' motivation through the lens of socially shared regulation of learning , journal =

    J. Investigating peer influence on collaborative group members' motivation through the lens of socially shared regulation of learning , journal =. 2025 , doi =

  6. [6]

    , title =

    Ching, Cynthia Carter and Kafai, Yasmin B. , title =. Teachers College Record: The Voice of Scholarship in Education , volume =. 2008 , URL=

  7. [7]

    Scientific Reports , volume=

    Engagement patterns of middle school students with AI teachable agents in mathematics learning , author=. Scientific Reports , volume=. 2025 , doi=

  8. [8]

    2015 , publisher=

    An introduction to agent-based modeling: modeling natural, social, and engineered complex systems with NetLogo , author=. 2015 , publisher=

  9. [9]

    Education Sciences , volume=

    Exploring education as a complex system: Computational educational research with multi-level agent-based modeling , author=. Education Sciences , volume=. 2024 , publisher=

  10. [10]

    Journal of International Technology and Information Management , volume=

    Agent-based modeling and simulation approaches in stem education research , author=. Journal of International Technology and Information Management , volume=. 2019 , doi=

  11. [11]

    Scientific Reports , volume=

    Engagement patterns of middle school students with AI teachable agents in mathematics learning , author=. Scientific Reports , volume=. 2025 , publisher=

  12. [12]

    Smart learning environments , volume=

    The effects of over-reliance on AI dialogue systems on students' cognitive abilities: a systematic review , author=. Smart learning environments , volume=. 2024 , publisher=

  13. [13]

    Frontiers in Artificial Intelligence , volume=

    Adolescents’ use and perceived usefulness of generative AI for schoolwork: exploring their relationships with executive functioning and academic achievement , author=. Frontiers in Artificial Intelligence , volume=. 2024 , publisher=

  14. [14]

    Proceedings of the National Academy of Sciences , volume=

    Generative AI without guardrails can harm learning: Evidence from high school mathematics , author=. Proceedings of the National Academy of Sciences , volume=. 2025 , publisher=

  15. [15]

    InfoTech Spectrum: Iraqi Journal of Data Science , volume=

    Evaluating the Effectiveness of AI Tools in Mathematical Modelling of Various Life Phenomena: A Proposed Approach , author=. InfoTech Spectrum: Iraqi Journal of Data Science , volume=. 2025 , DOI=

  16. [16]

    JoPhRA: Journal of Physics Research and It's Application , volume=

    Analysis of the Use of Gpt Chat to Improve Understanding of Physics Concepts , author=. JoPhRA: Journal of Physics Research and It's Application , volume=. 2025 , url=

  17. [17]

    Rapporto Nazionale INVALSI 2025: Presentazione , year =

  18. [18]

    Sage Open , volume=

    Investigating high school students’ attitudes toward the use of AI in education: Evidence from Cambodia , author=. Sage Open , volume=. 2025 , doi=

  19. [19]

    Abosaooda and Syaiba Ariffin and Osamah Mohammed Alyasiri , title =

    Hadeel N. Abosaooda and Syaiba Ariffin and Osamah Mohammed Alyasiri , title =. IAES International Journal of Artificial Intelligence (IJ-AI) , volume =. 2026 , keywords =

  20. [20]

    Teaching Mathematics with the Assistance of an AI Chatbot to Enhance Mathematical Thinking Skills for High School Students , volume=

    Nguyen and Van Doc, Nguyen and Hoai Nam, Nguyen Thi and Thanh, Ngo Tu and Minh Giam, Nguyen and , year=. Teaching Mathematics with the Assistance of an AI Chatbot to Enhance Mathematical Thinking Skills for High School Students , volume=. International Journal of Current Science Research and Review , publisher=. doi:10.47191/ijcsrr/v6-i12-102 , number=

  21. [21]

    Societies , VOLUME =

    Gerlich, Michael , TITLE =. Societies , VOLUME =. 2025 , NUMBER =

  22. [22]

    2025 , month =

    Tuveri, M and Sanna, A P and Cadoni, M , title =. 2025 , month =. doi:10.1088/1361-6404/ae21a6 , url =

  23. [23]

    2024 , eprint=

    Embracing AI in Education: Understanding the Surge in Large Language Model Use by Secondary Students , author=. 2024 , eprint=

  24. [24]

    L. S. Vygotsky , publisher =. Mind in Society: Development of Higher Psychological Processes , urldate =

  25. [25]

    , editor =

    Bruun, Jesper and Bearden, Ian G. , editor =. Time. PLoS ONE , volume =. doi:10.1371/journal.pone.0112775 , urldate =

  26. [26]

    Proceedings of the National Academy of Sciences , volume =

    Community Structure in Social and Biological Networks , author =. Proceedings of the National Academy of Sciences , volume =. doi:10.1073/pnas.122653799 , urldate =

  27. [27]

    Physical Review E , volume =

    Stochastic Blockmodels and Community Structure in Networks , author =. Physical Review E , volume =. doi:10.1103/PhysRevE.83.016107 , urldate =

  28. [28]

    Physical Review E (2004) https://doi.org/10.1103/physreve.69.026113

    Finding and Evaluating Community Structure in Networks , author =. Physical Review E , volume =. doi:10.1103/PhysRevE.69.026113 , urldate =

  29. [29]

    , year = 2014, month = jan, journal =

    Peixoto, Tiago P. , year = 2014, month = jan, journal =. Efficient. doi:10.1103/PhysRevE.89.012804 , urldate =

  30. [30]

    Physical Review Physics Education Research , volume =

    Long-Term Collaboration with Strong Friendship Ties Improves Academic Performance in Remote and Hybrid Teaching Modalities in High School Physics , author =. Physical Review Physics Education Research , volume =. doi:10.1103/PhysRevPhysEducRes.18.010146 , urldate =

  31. [31]

    Formation of Homophily in Academic Performance:

    Smirnov, Ivan and Thurner, Stefan , editor =. Formation of Homophily in Academic Performance:. PLOS ONE , volume =. doi:10.1371/journal.pone.0183473 , urldate =

  32. [32]

    , year = 2015, month = dec, pages =

    Traxler, Adrienne L. , year = 2015, month = dec, pages =. Community Structure in Introductory Physics Course Networks , booktitle =. doi:10.1119/perc.2015.pr.078 , urldate =

  33. [33]

    PLOS ONE , volume =

    A Description of Classroom Help Networks, Individual Network Position, and Their Associations with Academic Achievement , author =. PLOS ONE , volume =. doi:10.1371/journal.pone.0208173 , urldate =

  34. [34]

    On random graphs

    Erd. On random graphs. I , journal =. 1959 , volume =

  35. [35]

    Nature , langid =

    Collective Dynamics of `Small-World' Networks , author =. Nature , langid =

  36. [36]

    doi: https://doi.org/10.1016/0378-8733(83)90021-7

    Stochastic blockmodels: First steps , journal =. 1983 , issn =. doi:https://doi.org/10.1016/0378-8733(83)90021-7 , author =

  37. [37]

    Physical Review Physics Education Research , volume =

    Students' Network Integration as a Predictor of Persistence in Introductory Physics Courses , author =. Physical Review Physics Education Research , volume =. doi:10.1103/PhysRevPhysEducRes.13.010113 , urldate =

  38. [38]

    Physical Review Physics Education Research , volume =

    Network Positions in Active Learning Environments in Physics , author =. Physical Review Physics Education Research , volume =. doi:10.1103/PhysRevPhysEducRes.16.020129 , urldate =

  39. [39]

    Physical Review Physics Education Research , volume =

    Characterizing Active Learning Environments in Physics Using Network Analysis and Classroom Observations , author =. Physical Review Physics Education Research , volume =. doi:10.1103/PhysRevPhysEducRes.17.020136 , urldate =

  40. [40]

    Centola, Damon , year = 2010, month = sep, journal =. The. doi:10.1126/science.1185231 , urldate =

  41. [41]

    Comparison of Voter and

    Castellano, Claudio and Loreto, Vittorio and Barrat, Alain and Cecconi, Federico and Parisi, Domenico , year = 2005, month = jun, journal =. Comparison of Voter and. doi:10.1103/PhysRevE.71.066107 , urldate =

  42. [42]

    Random graphs as models of networks

    Random Graphs as Models of Networks , author =. doi:10.48550/arXiv.cond-mat/0202208 , urldate =. cond-mat/0202208 , publisher =

  43. [43]

    Gilbert, E. N. , year = 1959, journal =. Random. 2237458 , eprinttype =

  44. [44]

    Stochastic

    Barbillon, Pierre and Donnet, Sophie and Lazega, Emmanuel and. Stochastic. Journal of the Royal Statistical Society Series A: Statistics in Society , volume =. doi:10.1111/rssa.12193 , urldate =

  45. [45]

    The Annals of Applied Statistics , volume =

    Coauthorship and Citation Networks for Statisticians , author =. The Annals of Applied Statistics , volume =. doi:10.1214/15-AOAS896 , urldate =

  46. [46]

    Proceedings of the National Academy of Sciences , volume =

    The Structure of Scientific Collaboration Networks , author =. Proceedings of the National Academy of Sciences , volume =. doi:10.1073/pnas.98.2.404 , urldate =

  47. [47]

    Collaboration and

    Uzzi, Brian and Spiro, Jarrett , year = 2005, month = sep, journal =. Collaboration and. doi:10.1086/432782 , urldate =

  48. [48]

    Weeden, Kim and Cornwell, Benjamin , year = 2020, journal =. The. doi:10.15195/v7.a9 , urldate =

  49. [49]

    Proceedings of the National Academy of Sciences , volume =

    Classes of Small-World Networks , author =. Proceedings of the National Academy of Sciences , volume =. doi:10.1073/pnas.200327197 , urldate =

  50. [50]

    Small World Is Not Enough:

    Thiriot, Samuel , year = 2020, month = feb, number =. Small World Is Not Enough:. 2002.03717 , primaryclass =

  51. [51]

    Journal of Economic Dynamics and Control , volume =

    Network Structure and the Diffusion of Knowledge , author =. Journal of Economic Dynamics and Control , volume =

  52. [52]

    Coordination and Cooperation in Local, Random and Small World Networks:

    Cassar, Alessandra , year = 2007, month = feb, journal =. Coordination and Cooperation in Local, Random and Small World Networks:

  53. [53]

    Theoretical Description of Teaching-Learning Processes: A Multidisciplinary Approach , author =. Phys. Rev. Lett. , volume =. 2001 , month =

  54. [54]

    2021 , issn =

    An agent-based model for teaching–learning processes , journal =. 2021 , issn =. doi:https://doi.org/10.1016/j.physa.2020.125563 , author =

  55. [55]

    Agent-Based-Model of Students' Sociocognitive Learning Process in Acquiring Tiered Knowledge

    Koponen, Ismo T. Agent-Based-Model of Students' Sociocognitive Learning Process in Acquiring Tiered Knowledge. Modeling and Simulation of Social-Behavioral Phenomena in Creative Societies. 2019

  56. [56]

    2012 , doi =

    Academic Achievement and Its Impact on Friend Dynamics , journal =. 2012 , doi =

  57. [57]

    Dynamic friendship processes related to learning interest: Moderating by class-level social–emotional competency , journal =

    Qin, Kexin and Zhang, Yimei and Zhang, Tianshu and Wang, Yehui , keywords =. Dynamic friendship processes related to learning interest: Moderating by class-level social–emotional competency , journal =. 2025 , doi =

  58. [58]

    Virdis, Massimiliano , title =

  59. [59]

    2026 , month = may, howpublished =

  60. [60]

    and Celeux, G

    Biernacki, C. and Celeux, G. and Govaert, G. , journal=. Assessing a mixture model for clustering with the integrated completed likelihood , year=