pith. sign in

arxiv: 2604.08589 · v1 · submitted 2026-03-31 · 💻 cs.LG

EngageTriBoost: Predictive Modeling of User Engagement in Digital Mental Health Intervention Using Explainable Machine Learning

Ha Na Cho , Daniel Eisenberg , Cheryl King , Kai Zheng This is my paper

Pith reviewed 2026-05-13 23:19 UTC · model grok-4.3

classification 💻 cs.LG
keywords digital mental health interventionsuser engagement predictionensemble machine learningSHAP explainabilitycollege student mental healthpredictive modelingdropout predictionmotivational interviewing
0
0 comments X

The pith

An ensemble model called EngageTriBoost predicts user engagement in a digital mental health program for college students with up to 84 percent accuracy and uses SHAP to link emotional dysregulation and stigma to participation levels.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds an ensemble machine learning model to forecast whether at-risk college students will sign in and interact with counselors in the eBridge online program, which uses motivational interviewing to connect users to professional help. The model reaches 84 percent accuracy on behavioral and self-reported data from the eBridge dataset. SHAP explanations then rank the most influential features, showing emotional dysregulation and perceived stigma as strong drivers of low or high engagement. A reader would care because high dropout rates currently limit how many people digital tools can actually reach. If the predictions and explanations hold, program designers could adjust onboarding or content to keep more users active and thereby increase the number who receive needed services.

Core claim

The central claim is that the EngageTriBoost ensemble achieves up to 84 percent accuracy when predicting engagement, defined as sign-ins and counselor interactions, and that SHAP analysis applied afterward identifies emotional dysregulation and perceived stigma as the factors with the largest effects on whether users stay with the digital mental health intervention.

What carries the argument

The EngageTriBoost ensemble model paired with Shapley Additive exPlanations (SHAP) to generate interpretable feature rankings from user sign-in and interaction data.

If this is right

  • Programs could use early predictions to send targeted reminders or simplified onboarding to users flagged as likely to disengage.
  • Design changes that reduce perceived stigma or support emotional regulation could raise overall sign-in and interaction rates.
  • Resource allocation in counseling services could shift toward users the model identifies as high-engagement prospects.
  • The same pipeline of ensemble prediction plus SHAP ranking could be reused on data from similar motivational-interviewing platforms.
  • Higher sustained engagement would increase the fraction of at-risk students who complete the referral to in-person professional care.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Testing the model in a randomized trial where one arm receives SHAP-guided adjustments could reveal whether the identified factors are modifiable enough to move engagement numbers.
  • Similar explainable models might transfer to other digital health domains such as chronic-disease self-management apps where dropout is also common.
  • If emotional dysregulation ranks high across datasets, it suggests screening for that trait at enrollment could become standard for improving retention.
  • Longer-term follow-up data would show whether the users the model predicts as engaged actually experience better mental-health outcomes than low-engagement users.

Load-bearing premise

The study assumes that the behavioral patterns and self-reported features observed in the eBridge dataset of college students are representative enough to generalize and that the SHAP rankings point to drivers that can be changed rather than fixed correlations.

What would settle it

Apply the trained model to a fresh, independent set of users from another digital mental health program and measure whether accuracy falls below 70 percent or whether the top SHAP-ranked features shift to different variables.

Figures

Figures reproduced from arXiv: 2604.08589 by Cheryl King, Daniel Eisenberg, Ha Na Cho, Kai Zheng.

Figure 1
Figure 1. Figure 1: Pipeline for predicting digital mental health engagement outcomes, starting from eBridge trial data preprocessing to modeling login and message posting outcomes using an ensemble approach with performance evaluation and SHAP explainability [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: SHAP explanations of the top ten features influencing user login behavior. (a) Mean absolute SHAP values, representing the global feature imporatnce. (b) Summary plot feature value (red=high, blue=low) and SHAP impact (right=increased, left=decreased login). (c) Decision plot visualizing the cumulative contribution of key features in representative individual prediction [PITH_FULL_IMAGE:figures/full_fig_p… view at source ↗
Figure 2
Figure 2. Figure 2: a. shows that higher chronic pain values were associated with increased SHAP attribution toward login predictions, whereas lower pain values were associated with decreased attribution toward login. This pattern is visually reinforced by the rightward shift and red coloration in the SHAP summary bars for chronic pain, indicating positive SHAP values pushing the model toward a login output. Elevated suicidal… view at source ↗
Figure 3
Figure 3. Figure 3: SHAP explanations for top ten features influencing message posting behavior, presented using the same layout and interpretation as in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
read the original abstract

Mental health challenges among young adults, are on the rise, necessitating effective solutions such as digital mental health interventions (DMHIs). Despite their promise, DMHIs face significant adoption barriers, including low initial uptake and high dropout rates. This study leverages machine learning (ML) to analyze behavioral patterns of users of a DMHI, eBridge, designed to increase the utilization of professional mental health services among at-risk college students through motivational interviewing-based online counseling. Our ensemble model, EngageTriBoost, achieved up to 84% accuracy in predicting engagement, measured by sign-ins and counselor interactions. We then applied the Shapley Additive exPlanations (SHAP) analysis which provided clear, interpretable insights into key factors influencing user engagement such as emotional dysregulation and perceived stigma, highlighting their critical effect on DMHI adoption. This study demonstrates the power of explainable ML for better understanding user engagement with DMHI to improve their adoption and achievable impact on mental health outcomes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript presents EngageTriBoost, an ensemble machine learning model for predicting user engagement (measured by sign-ins and counselor interactions) in the eBridge digital mental health intervention for at-risk college students. It reports up to 84% accuracy and applies SHAP analysis to identify interpretable factors such as emotional dysregulation and perceived stigma as key influences on engagement and DMHI adoption.

Significance. If the performance and interpretability claims are supported by rigorous validation, the work could aid in identifying barriers to DMHI uptake and informing targeted improvements. The use of an ensemble approach combined with post-hoc explainability is a positive step toward actionable insights in mental health applications. However, missing methodological details and the correlational nature of SHAP limit the strength of conclusions about critical effects or generalizability.

major comments (2)
  1. [Abstract] Abstract: The central performance claim of 'up to 84% accuracy' is stated without any information on dataset size (e.g., number of users or sessions in eBridge), validation strategy (train-test split, cross-validation folds), baseline comparisons, or handling of class imbalance. This absence makes it impossible to determine whether the result exceeds trivial baselines or supports the predictive modeling contribution.
  2. [SHAP Analysis] SHAP Analysis section: The manuscript states that SHAP 'provided clear, interpretable insights into key factors influencing user engagement such as emotional dysregulation and perceived stigma, highlighting their critical effect on DMHI adoption.' SHAP values quantify additive feature contributions to model predictions on the observed distribution and do not perform causal inference or counterfactual reasoning. No mention is made of causal methods, randomized elements in the eBridge design, or confounder sensitivity checks, so the leap from correlation to 'critical effect' is unsupported.
minor comments (1)
  1. [Abstract] The abstract and introduction could more explicitly define the engagement labels (sign-ins vs. counselor interactions) and any preprocessing steps applied to the behavioral and self-reported features.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which help strengthen the clarity and precision of our work. We address each major comment point by point below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central performance claim of 'up to 84% accuracy' is stated without any information on dataset size (e.g., number of users or sessions in eBridge), validation strategy (train-test split, cross-validation folds), baseline comparisons, or handling of class imbalance. This absence makes it impossible to determine whether the result exceeds trivial baselines or supports the predictive modeling contribution.

    Authors: We agree that the abstract should be self-contained to allow readers to evaluate the performance claim. The full manuscript details the eBridge dataset size, 5-fold cross-validation strategy, baseline comparisons (including logistic regression and random forest), and class imbalance handling via weighted objectives. To address the referee's concern, we will revise the abstract to concisely report the sample size, validation approach, and that accuracy exceeds the majority-class baseline. revision: yes

  2. Referee: [SHAP Analysis] SHAP Analysis section: The manuscript states that SHAP 'provided clear, interpretable insights into key factors influencing user engagement such as emotional dysregulation and perceived stigma, highlighting their critical effect on DMHI adoption.' SHAP values quantify additive feature contributions to model predictions on the observed distribution and do not perform causal inference or counterfactual reasoning. No mention is made of causal methods, randomized elements in the eBridge design, or confounder sensitivity checks, so the leap from correlation to 'critical effect' is unsupported.

    Authors: We concur that SHAP quantifies correlational feature contributions and does not support causal claims. The eBridge data is observational with no randomized elements or causal methods applied. We will revise the wording in the abstract and SHAP section from 'critical effect' to 'key associations' to accurately reflect the correlational nature of the results. We will also add a limitations paragraph noting that SHAP insights are associative and that causal inference would require additional methods or study designs. revision: yes

Circularity Check

0 steps flagged

No significant circularity in the modeling or explanation pipeline

full rationale

The paper describes a standard supervised ML workflow: features from the eBridge dataset are used to train the EngageTriBoost ensemble to predict engagement (sign-ins and counselor interactions), with accuracy evaluated on held-out data. SHAP is then applied post-hoc to attribute contributions to the trained model's outputs. No derivation step reduces the target variable or predictions to quantities defined by the fitted parameters themselves, no self-citation chain is load-bearing for the core claims, and no ansatz or uniqueness theorem is smuggled in. The reported accuracy and feature attributions are computed independently of the input definitions, making the chain self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review yields no identifiable free parameters, axioms, or invented entities; the work rests on standard machine-learning assumptions such as feature relevance and data representativeness that are not detailed here.

pith-pipeline@v0.9.0 · 5473 in / 1224 out tokens · 65263 ms · 2026-05-13T23:19:06.501647+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    Our ensemble model, EngageTriBoost, achieved up to 84% accuracy in predicting engagement... We then applied the Shapley Additive exPlanations (SHAP) analysis which provided clear, interpretable insights into key factors influencing user engagement such as emotional dysregulation and perceived stigma.

  • IndisputableMonolith/Foundation/ArithmeticFromLogic.lean LogicNat recovery unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    ETB integrates XGBoost, LightGBM, and CatBoost as base learners with logistic regression as a meta-learner, tuned through cross-validation.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

  1. [1]

    Online suicide risk screening and intervention with college students: a pilot randomized controlled trial,

    C. A. King, D. Eisenberg, K. Zheng, E. Czyz, A. Kramer, A. Horwitz, and S. Chermack, “Online suicide risk screening and intervention with college students: a pilot randomized controlled trial,” J Consult Clin Psychol, vol. 83, no. 3, pp. 630-6, 2015. [9] C. A. King, D. Eisenberg, and J. Pistorello, “Electronic bridge to mental health for college students:...

    work page 2015

  2. [2]

    Longitudinal patterns of engagement and clinical outcomes: results from a therapist-supported digital mental health intervention,

    K. Aschbacher, L. M. Rivera, S. Hornstein, B. W. Nelson, V. Forman-Hoffman, and N. Peiper, “Longitudinal patterns of engagement and clinical outcomes: results from a therapist-supported digital mental health intervention,” Psychosom Med, 2023. [29] Y. Huang, H. Liu, S. Li, W. Wang, and Z. Zhou, “Effective prediction and important counseling experience for...

    work page 2023