ProPACT: A Proactive AI-Driven Adaptive Collaborative Tutor for Pair Programming
Pith reviewed 2026-05-08 18:53 UTC · model grok-4.3
The pith
ProPACT uses forecasts of joint attention and effort to deliver proactive scaffolds that improve pair programming outcomes.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
ProPACT constructs a multimodal dyadic learner model based on Joint Visual Attention (JVA), Joint Mental Effort (JME), and individual mental effort, and employs an XGBoost-based forecasting model to predict emerging suboptimal collaboration states up to 30 seconds in advance. These predictions drive a hierarchical adaptive policy that delivers minimally intrusive scaffolds while fading support during productive collaboration. A within-subject study with 26 pair-programming dyads shows that proactive feedback significantly improves debugging success, task efficiency, feedback uptake, and post-intervention gains in JVA and JME.
What carries the argument
The XGBoost forecasting model operating on the multimodal dyadic learner model to predict suboptimal states 30 seconds ahead and trigger a fading scaffold policy.
If this is right
- Proactive interventions raise debugging success rates compared with reactive or no feedback.
- Task completion becomes more efficient when scaffolds arrive before coordination breaks down.
- Learners show higher uptake of the feedback itself under the proactive schedule.
- Gains in joint visual attention and joint mental effort persist after the intervention ends.
Where Pith is reading between the lines
- The same forecasting-plus-fading approach could be tested in other real-time team tasks such as remote design reviews or surgical simulations.
- If prediction windows can be extended or combined with additional sensors, the system might support larger groups without increasing intrusion.
- The emphasis on fading support suggests a pathway for gradually transferring regulation skills from the AI to the human pairs.
Load-bearing premise
The multimodal dyadic learner model based on joint visual attention, joint mental effort, and individual mental effort, together with the XGBoost forecasting model, can accurately predict emerging suboptimal collaboration states up to 30 seconds in advance.
What would settle it
A follow-up experiment in which prediction accuracy falls below usable levels and the measured gains in debugging success, efficiency, and post-intervention JVA/JME disappear.
Figures
read the original abstract
Effective pair programming depends on coordination of attention, cognitive effort, and joint regulation over time, yet most adaptive learning systems remain individual-centric and reactive. This paper introduces ProPACT, a proactive AI-driven adaptive collaborative tutor that treats collaboration itself as the object of instruction. ProPACT constructs a multimodal dyadic learner model based on Joint Visual Attention (JVA), Joint Mental Effort (JME), and individual mental effort, and employs an XGBoost-based forecasting model to predict emerging suboptimal collaboration states up to 30 seconds in advance. These predictions drive a hierarchical adaptive policy that delivers minimally intrusive scaffolds while fading support during productive collaboration. A within-subject study with 26 pair-programming dyads shows that proactive feedback significantly improves debugging success, task efficiency, feedback uptake, and post-intervention gains in JVA and JME, demonstrating the potential of forecast-driven dyadic adaptivity for real-time collaborative learning regulation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces ProPACT, a proactive AI-driven adaptive collaborative tutor for pair programming. It constructs a multimodal dyadic learner model using Joint Visual Attention (JVA), Joint Mental Effort (JME), and individual mental effort, then applies an XGBoost-based forecasting model to predict emerging suboptimal collaboration states up to 30 seconds ahead. These forecasts drive a hierarchical adaptive policy that delivers minimally intrusive scaffolds and fades support during productive periods. A within-subject study with 26 pair-programming dyads reports that proactive feedback yields significant gains in debugging success, task efficiency, feedback uptake, and post-intervention improvements in JVA and JME.
Significance. If the forecasting model is shown to be reliable, the work advances the field by demonstrating forecast-driven dyadic adaptivity rather than reactive individual scaffolding. The within-subject design with 26 dyads is a clear empirical strength, enabling direct comparison of proactive versus baseline conditions. The integration of multimodal signals (JVA/JME) into a real-time policy is a substantive contribution to collaborative learning systems.
major comments (2)
- The description of the XGBoost forecasting model (Section 4.2) reports no performance metrics whatsoever: no accuracy, AUC, precision-recall at varying lead times, cross-validation scheme, or comparison against a reactive baseline. This is load-bearing for the central claim that predictions enable effective proactive scaffolding; without these numbers the observed study gains could be produced by any feedback rather than by its forecast-driven timing.
- Results section (Section 5): the abstract and text assert statistically significant improvements in debugging success, efficiency, uptake, and JVA/JME gains, yet supply no p-values, effect sizes, confidence intervals, or details on data-handling rules (normality, multiple-comparison correction, outlier policy). These omissions prevent assessment of whether the within-subject findings robustly support the proactive-policy conclusion.
minor comments (2)
- Figure 3 (policy diagram) and the accompanying text use inconsistent terminology for 'suboptimal states' versus 'emerging suboptimal collaboration'; a single definition and cross-reference would improve clarity.
- The participant demographics and task description (Section 5.1) are brief; adding a table of dyad characteristics and exact task instructions would aid reproducibility.
Simulated Author's Rebuttal
We thank the referee for the thoughtful and constructive review. The comments identify key areas where additional detail will strengthen the manuscript's claims regarding the forecasting model and statistical reporting. We address each major comment below and will incorporate revisions as noted.
read point-by-point responses
-
Referee: The description of the XGBoost forecasting model (Section 4.2) reports no performance metrics whatsoever: no accuracy, AUC, precision-recall at varying lead times, cross-validation scheme, or comparison against a reactive baseline. This is load-bearing for the central claim that predictions enable effective proactive scaffolding; without these numbers the observed study gains could be produced by any feedback rather than by its forecast-driven timing.
Authors: We agree that detailed performance metrics for the XGBoost model are necessary to substantiate the proactive nature of the scaffolding. The current manuscript emphasizes the end-to-end system and empirical outcomes but does not include these evaluations. In the revised manuscript, we will add accuracy, AUC-ROC, precision-recall curves across lead times (0-30s), the cross-validation scheme (e.g., time-series split or dyad-level CV), and a direct comparison to a reactive baseline that triggers on current rather than forecasted states. This addition will demonstrate that the observed benefits derive from reliable forecast-driven timing rather than generic feedback. revision: yes
-
Referee: Results section (Section 5): the abstract and text assert statistically significant improvements in debugging success, efficiency, uptake, and JVA/JME gains, yet supply no p-values, effect sizes, confidence intervals, or details on data-handling rules (normality, multiple-comparison correction, outlier policy). These omissions prevent assessment of whether the within-subject findings robustly support the proactive-policy conclusion.
Authors: We acknowledge that the statistical reporting in Section 5 is incomplete. The within-subject design with 26 dyads used paired tests (t-tests or Wilcoxon signed-rank as determined by Shapiro-Wilk normality checks), with Bonferroni correction applied for the four primary outcomes. In the revision, we will report exact p-values, Cohen's d effect sizes, 95% confidence intervals for all key metrics, and explicit details on normality testing, outlier policy (e.g., >3 SD from mean), and multiple-comparison correction. This will enable readers to fully assess the robustness of the proactive-policy effects. revision: yes
Circularity Check
No significant circularity; claims rest on empirical study results
full rationale
The paper's core contribution is an empirical within-subject study with 26 dyads demonstrating gains in debugging success, efficiency, feedback uptake, and post-intervention JVA/JME improvements from proactive scaffolds. The multimodal dyadic model and XGBoost forecaster are presented as implemented components whose outputs drive the policy, but no derivation chain, equations, or self-citations reduce any central claim to a fitted parameter or self-definition. The absence of reported XGBoost validation metrics (accuracy, lead-time performance) is a separate evidence gap rather than circularity; the study results remain independent of any internal reduction to inputs.
Axiom & Free-Parameter Ledger
free parameters (1)
- XGBoost hyperparameters and prediction thresholds
axioms (1)
- domain assumption JVA, JME, and individual mental effort are valid proxies for dyadic collaboration quality
Lean theorems connected to this paper
-
Cost.FunctionalEquation / J-costwashburn_uniqueness_aczel (no parallel: thresholds are empirical, not ratio-symmetric) unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
A two–standard-deviation (2SD) criterion relative to each participant's resting baseline was applied, with values above +2SD classified as High, below −2SD as Low...
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Sharma, K., Molinari, G.: What happens when collaborators are not in-synch? In: Proceedings of the 16th International Conference on Computer-Supported Collaborative Learning-CSCL 2023, Pp. 99-106 (2023). International Society of the Learning Sciences
work page 2023
-
[2]
In: Proceedings of the 2017 Acm Sigcse Technical Symposium on Computer Science Education, pp
Rodríguez, F.J., Price, K.M., Boyer, K.E.: Exploring the pair programming process: Characteristics of effective collaboration. In: Proceedings of the 2017 Acm Sigcse Technical Symposium on Computer Science Education, pp. 507–512 (2017) 11
work page 2017
-
[3]
British Journal of Educational Technology51(6), 2391–2406 (2020)
Järvelä, S., Gašević, D., Seppänen, T., Pechenizkiy, M., Kirschner, P.A.: Bridging learning sciences, machine learning and affective computing for understanding cognition and affect in collaborative learning. British Journal of Educational Technology51(6), 2391–2406 (2020)
work page 2020
-
[4]
International journal of computer-supported collaborative learning14(4), 425–441 (2019)
Järvelä, S., Järvenoja, H., Malmberg, J.: Capturing the dynamic and cyclical nature of regulation: Methodological progress in understanding socially shared regulation in learning. International journal of computer-supported collaborative learning14(4), 425–441 (2019)
work page 2019
-
[5]
Journal of Computer Assisted Learning39(5), 1714–1731 (2023)
Liu, C.-Y., Li, W., Huang, J.-Y., Lei, L.-Y., Zhang, P.-R.: Collaborative pro- gramming based on social shared regulation: An approach to improving students’ programming achievements and group metacognition. Journal of Computer Assisted Learning39(5), 1714–1731 (2023)
work page 2023
-
[6]
Frontline Learning Research8(2), 1–34 (2020)
Bakhtiar, A., Hadwin, A.F.: Dynamic interplay between modes of regulation during motivationally challenging episodes in collaboration. Frontline Learning Research8(2), 1–34 (2020)
work page 2020
-
[7]
Sharma,K., Olsen, J.K.:What bringsstudents together?: Investigating thecausal relationship between joint mental effort and joint visual attention. In: Proceed- ings of the 15th International Conference on Computer-Supported Collaborative Learning-CSCL 2022, Pp. 131-138 (2022). International Society of the Learning Sciences
work page 2022
-
[8]
Educational technology research and development72(3), 1383–1418 (2024)
Ke, F., Liu, R., Sokolikj, Z., Dahlstrom-Hakki, I., Israel, M.: Using eye-tracking in education: review of empirical research and technology. Educational technology research and development72(3), 1383–1418 (2024)
work page 2024
-
[9]
In: 2021 IEEE Virtual Reality and 3D User Interfaces (VR), pp
Bueno-Vesga, J.A., Xu, X., He, H.: The effects of cognitive load on engagement in a virtual reality learning environment. In: 2021 IEEE Virtual Reality and 3D User Interfaces (VR), pp. 645–652 (2021). IEEE
work page 2021
-
[10]
Polat, H., Kayaduman, H., Taş, N., Battal, A., Kaban, A., Bayram, E.: Learning declarative and procedural knowledge through instructor-present videos: learn- ing effectiveness, mental effort, and visual attention allocation: H. polat et al. Educational technology research and development73(6), 3479–3513 (2025)
work page 2025
-
[11]
International Society of the Learning Sciences, Proceedings
Sharma, K., Olsen, J., Verma, H., Caballero, D., Jermann, P.: Challenging Joint Visual Attention as a Proxy for Collaborative Performance: ISLS Annual Meet- ing 2021 (virtual): International Society of the Learning Sciences. International Society of the Learning Sciences, Proceedings. 91–98 (2021) (2021)
work page 2021
-
[12]
Interacting with Computers33(2), 115–133 (2021) 12
D’angelo, S., Schneider, B.: Shared gaze visualizations in collaborative inter- actions: Past, present and future. Interacting with Computers33(2), 115–133 (2021) 12
work page 2021
-
[13]
Sharma, K., Giannakos, M.: Multimodal data capabilities for learning: What can multimodal data tell us about learning? British Journal of Educational Technology51(5), 1450–1484 (2020)
work page 2020
-
[14]
In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp
Spikol, D., Prieto, L.P., Rodríguez-Triana, M.J., Worsley, M., Ochoa, X., Cukurova, M., Vogel, B., Ruffaldi, E., Ringtved, U.L.: Current and future multimodal learning analytics data challenges. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 518–519 (2017)
work page 2017
- [15]
-
[16]
Wohltjen, S., Wheatley, T.: Interpersonal eye-tracking reveals the dynamics of interacting minds. Front. Hum. Neurosci. 18 (2024)
work page 2024
-
[17]
British Journal of Educational Technology55(5), 1900–1925 (2024)
Yan,L.,Echeverria,V.,Jin,Y.,Fernandez-Nieto,G.,Zhao,L.,Li,X.,Alfredo,R., Swiecki, Z., Gašević, D., Martinez-Maldonado, R.: Evidence-based multimodal learning analytics for feedback and reflection in collaborative learning. British Journal of Educational Technology55(5), 1900–1925 (2024)
work page 1900
-
[18]
British Journal of Educational Technology56(2), 712–733 (2025)
Edwards, J., Nguyen, A., Lämsä, J., Sobocinski, M., Whitehead, R., Dang, B., Roberts, A.-S., Järvelä, S.: Human-ai collaboration: Designing artificial agents to facilitate socially shared regulation among learners. British Journal of Educational Technology56(2), 712–733 (2025)
work page 2025
-
[19]
In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, pp
Moreno, G.A., Cámara, J., Garlan, D., Schmerl, B.: Proactive self-adaptation under uncertainty: a probabilistic model checking approach. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, pp. 1–12 (2015)
work page 2015
-
[20]
Journal of Learning Analytics7(1), 59–71 (2020)
Kent, C., Cukurova, M.: Investigating collaboration as a process with theory- driven learning analytics. Journal of Learning Analytics7(1), 59–71 (2020)
work page 2020
-
[21]
Frontiers in Artificial Intelligence, 6, 1151391 (2023)
Mallik, S., Gangopadhyay, A.: Proactive and reactive engagement of artificial intelligence methods for education: a review. Frontiers in Artificial Intelligence, 6, 1151391 (2023)
work page 2023
-
[22]
Scientific Reports15(1), 39797 (2025)
Alkan, B.B., Kuzucuk, S., Alkan, N., Sinan, A.: Using machine learning to pre- dict student outcomes for early intervention and formative assessment. Scientific Reports15(1), 39797 (2025)
work page 2025
-
[23]
British Journal of Educational Technology55(4), 1398–1436 (2024) 13
Sharma, K., Nguyen, A., Hong, Y.: Self-regulation and shared regulation in col- laborativelearninginadaptivedigitallearningenvironments:Asystematicreview of empirical studies. British Journal of Educational Technology55(4), 1398–1436 (2024) 13
work page 2024
-
[24]
In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp
Duchowski, A.T., Krejtz, K., Biele, C., Niedzielska, A., Kiefer, P., Raubal, M., Giannopoulos, I.: The index of pupillary activity: Measuring cognitive load vis- à-vis task difficulty with pupil oscillation. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2018)
work page 2018
-
[25]
International Journal of Computer-Supported Collaborative Learning 13(3), 241–261 (2018)
Schneider, B., Sharma, K., Cuendet, S., Dillenbourg, P.: Leveraging mobile eye- trackers to capture joint visual attention in co-located collaborative learning groups. International Journal of Computer-Supported Collaborative Learning 13(3), 241–261 (2018)
work page 2018
-
[26]
Sharma, K., Molinari, G.: What happens when collaborators are not in-synch? In: Proceedings of the 16th International Conference on Computer-Supported Collaborative Learning (CSCL 2023), pp. 99–106 (2023)
work page 2023
-
[27]
Frontiers in psychology5, 510 (2014)
Coco, M.I., Dale, R.: Cross-recurrence quantification analysis of categorical and continuous time series: an r package. Frontiers in psychology5, 510 (2014)
work page 2014
-
[28]
Efficiency, and Processes in Brownfield Programming Tasks.(06 2025) (2025)
Shihab, M., Hundhausen, C., Tariq, A., Haque, S., Qiao, Y., Mulanda, B.: The effects of github copilot on computing students’ programming effectiveness. Efficiency, and Processes in Brownfield Programming Tasks.(06 2025) (2025)
work page 2025
-
[29]
In: 2025 5th International Conference on Artificial Intelligence and Education (ICAIE), pp
Choi, W.C., Peng, J., Choi, I.C., Lei, H., Lam, L.C., Chang, C.I.: Improving young learners with copilot: The influence of large language models (llms) on cognitive load and self-efficacy in k-12 programming education. In: 2025 5th International Conference on Artificial Intelligence and Education (ICAIE), pp. 284–288 (2025). IEEE
work page 2025
-
[30]
In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, pp
Jermann, P., Nüssli, M.-A.: Effects of sharing text selections on gaze cross- recurrence and interaction quality in a pair programming task. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, pp. 1125–1134 (2012)
work page 2012
-
[31]
In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp
D’Angelo, S., Begel, A.: Improving communication between pair programmers using shared gaze awareness. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 6245–6290 (2017)
work page 2017
-
[32]
International Journal of Computer-Supported Collaborative Learning15(4), 469–498 (2020)
Hayashi, Y.: Gaze awareness and metacognitive suggestions by a pedagogical conversational agent: an experimental investigation on interventions to sup- port collaborative learning process and performance. International Journal of Computer-Supported Collaborative Learning15(4), 469–498 (2020)
work page 2020
-
[33]
Marwan,S.,Dombe,A.,Price,T.W.:Unproductivehelp-seekinginprogramming: What it is and how to address it. In: Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education, pp. 54–60 (2020)
work page 2020
-
[34]
Xie, H., Wang, F., Hao, Y., Chen, J., An, J., Wang, Y., Liu, H.: The more total cognitive load is reduced by cues, the better retention and transfer of multimedia 14 learning: A meta-analysis and two meta-regression analyses. PloS one12(8), 0183884 (2017)
work page 2017
-
[35]
British Journal of Educational Technology54(5), 1057–1076 (2023)
Järvelä, S., Nguyen, A., Hadwin, A.: Human and artificial intelligence collabo- ration for socially shared regulation in learning. British Journal of Educational Technology54(5), 1057–1076 (2023)
work page 2023
-
[36]
In: Proceedings of the 6th ACM Conference on Conversational User Interfaces, pp
Edwards, J., Nguyen, A., Sobocinski, M., Lämsä, J., De Araujo, A., Dang, B., Whitehead, R., Roberts, A.-S., Kaarlela, M., Jarvela, S.: Mai-a proactive speech agent for metacognitive mediation in collaborative learning. In: Proceedings of the 6th ACM Conference on Conversational User Interfaces, pp. 1–5 (2024)
work page 2024
-
[37]
In: Proceedings of the 16th International Conference of the Learning Sciences-ICLS 2022, Pp
Nguyen, A., Järvelä, S., Wang, Y., Rosé, C.P.: Exploring socially shared regula- tion with an ai deep learning approach using multimodal data. In: Proceedings of the 16th International Conference of the Learning Sciences-ICLS 2022, Pp. 527-534 (2022). International Society of the Learning Sciences
work page 2022
-
[38]
Education and Information Technologies, 1–39 (2025)
Kim, J., Detrick, R., Yu, S., Song, Y., Bol, L., Li, N.: Socially shared regulation of learning and artificial intelligence: Opportunities to support socially shared regulation. Education and Information Technologies, 1–39 (2025)
work page 2025
-
[39]
Assessment & Evaluation in Higher Education50(1), 127–143 (2025)
Wood, J., Pitt, E.: Empowering agency through learner-orchestrated self- generated feedback. Assessment & Evaluation in Higher Education50(1), 127–143 (2025)
work page 2025
-
[40]
Li, M., Wilson, J.: Ai-integrated scaffolding to enhance agency and creativity in k-12 english language learners: A systematic review. Information16(7), 519 (2025)
work page 2025
-
[41]
Electronic Journal of e-Learning23(4), 126–141 (2025) 15
Rakhmetov,M.,Sadvakassova,A.,Saltanova,G.,Kuanbayeva,B.,Zhusupkalieva, G.: Evaluation of an ai-based feedback system for enhancing self-regulated learning in digital education platforms. Electronic Journal of e-Learning23(4), 126–141 (2025) 15
work page 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.