Developing an Effective and Automated Patient Engagement Estimator for Telehealth: A Machine Learning Approach

We discuss MET, a learning-based algorithm proposed for perceiving a patient's level of engagement during telehealth sessions. We leverage latent vectors corresponding to Affective and Cognitive features frequently used in psychology literature to understand a person's level of engagement in a semi-supervised GAN-based framework. We showcase the efficacy of this method from the perspective of mental health and more specifically how this can be leveraged for a better understanding of patient engagement during telemental health sessions. To further the development of similar technologies that can be useful for telehealth, we also plan to release a dataset MEDICA containing 1299 video clips, each 3 seconds long and show experiments on the same. Our framework reports a 40% improvement in RMSE (Root Mean Squared Error) over state-of-the-art methods for engagement estimation. In our real-world tests, we also observed positive correlations between the working alliance inventory scores reported by psychotherapists. This indicates the potential of the proposed model to present patient engagement estimations that aligns well with the engagement measures used by psychotherapists.