pith. sign in

arxiv: 2604.11551 · v2 · submitted 2026-04-13 · 💻 cs.HC · cs.SE

Participation and Power: A Case Study of Using Ecological Momentary Assessment to Engage Adolescents in Academic Research

Pith reviewed 2026-05-10 15:43 UTC · model grok-4.3

classification 💻 cs.HC cs.SE
keywords ecological momentary assessmentadolescent researchyouth-centered designgamificationprivacy concernsresearch ethicshuman-computer interactionengagement
0
0 comments X

The pith

Youth-centered EMA design with gamification sustains adolescent engagement in mental health research while exposing technical and privacy challenges.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a youth-centered Ecological Momentary Assessment platform and tests it in a longitudinal case study with adolescent twins examining mental health and sleep. Interviews with the research team show how specific design choices shaped onboarding, ongoing participation, risk monitoring, and data handling. Positive outcomes included sustained teen involvement through app features and easier administration via the web portal. Negative outcomes included instability that triggered parent privacy worries and rigid data structures that complicated usage metadata analysis. The authors distill these observations into interaction design guidelines meant to prioritize youth agency alongside ethical and research needs.

Core claim

Through interviews with the research team in a longitudinal EMA study of adolescent twins, the authors found that a teen-centered mobile app with gamified elements maintained participant engagement, a centralized web dashboard streamlined oversight and risk monitoring, yet technical instability generated parent privacy concerns and inflexible data structures limited researchers' ability to examine raw usage metadata.

What carries the argument

The youth-centered EMA platform (mobile app plus web researcher dashboard) evaluated via post-study interviews with the research team on its effects across onboarding, engagement, monitoring, and data interpretation.

If this is right

  • Similar youth-centered features could improve retention rates in other longitudinal adolescent health studies.
  • Centralized researcher dashboards can reduce administrative burden when managing multiple participants.
  • Technical stability must be prioritized to prevent downstream privacy concerns from parents.
  • Flexible data export options are needed to support deeper analysis of engagement patterns without extra privacy risks.
  • The resulting guidelines offer a practical starting point for balancing participant agency with study requirements.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Extending the same design principles to non-twin adolescents or different health topics could test whether the engagement benefits hold beyond this specific sample.
  • Incorporating participant-controlled data export features might further shift power toward youth while still meeting research needs.
  • The observed privacy concerns point to a broader need for EMA tools that let families preview exactly what metadata is collected before consent.
  • These platform lessons may connect to ongoing work on ethical data practices in other digital youth interventions.

Load-bearing premise

That interviews with the research team in this single twin-based case study are enough to link specific design choices to engagement and privacy outcomes and to generate guidelines usable in other adolescent EMA settings.

What would settle it

A replication study in a different adolescent population that applies the proposed guidelines yet observes no sustained engagement gains or continued privacy and data-analysis problems.

Figures

Figures reproduced from arXiv: 2604.11551 by Elmira Rashidi, Karla Badillo-Urquiola, Ozioma C. Oguine, Pamela J. Wisniewski.

Figure 1
Figure 1. Figure 1: Original EMA app (a) and modified case study app (b) key feature differences. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Web portal dashboard of the 30-Day EMA diary tool. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
read the original abstract

Ecological Momentary Assessment (EMA) is widely used to study adolescents' experiences; yet, how the design of EMA platforms shapes engagement, research practices, and power dynamics in youth studies remains under-examined. We developed a youth-centered EMA platform prioritizing youth engagement and researcher support, and evaluated it through a case study on a longitudinal investigation with adolescent twins focused on mental health and sleep behavior. Interviews with the research team examined how the platform design choices shaped participant onboarding, sustained engagement, risk monitoring, and data interpretation. The app's teen-centered design and gamified features sustained teen engagement, while the web portal streamlined administrative oversight through a centralized dashboard. However, technical instability and rigid data structures created significant hurdles, leading to privacy concerns among parents and complicating the researchers' ability to analyze raw usage metadata. We provide actionable interaction design guidelines for developing EMA platforms that prioritize youth agency, ethical practice, and research goals.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript presents a qualitative case study of a youth-centered EMA platform developed for a longitudinal study with adolescent twins examining mental health and sleep behavior. Drawing on interviews with the research team, it reports that teen-centered design and gamification features supported sustained engagement and that a web portal aided oversight, while technical instability and rigid data structures led to parental privacy concerns and hindered metadata analysis; the paper concludes with actionable design guidelines for EMA platforms emphasizing youth agency and ethical practice.

Significance. If the reported links between specific design choices and observed outcomes hold, the work contributes to HCI research on participatory and ethical technology design for adolescents by surfacing power dynamics in EMA studies and offering practical guidelines. The single-case focus on twins and mental health limits immediate generalizability but could still guide future platform development if the interpretive claims are better substantiated.

major comments (2)
  1. [Case study evaluation and findings] The central claims attributing sustained teen engagement to gamified/teen-centered features and privacy hurdles to technical instability/rigid structures (abstract and case study findings) rest exclusively on interviews with the research team. This leaves causal attributions vulnerable to self-report bias, as alternative explanations (e.g., twin-specific dynamics or external events) are not ruled out and no direct teen/parent data or raw usage logs are described to corroborate the effects.
  2. [Discussion and guidelines] The leap from this single twin-based case study to general 'actionable interaction design guidelines' for other adolescent EMA contexts (discussion section) overstates the evidence base, given the specific sample characteristics and lack of cross-context validation or falsification attempts.
minor comments (1)
  1. [Methods] The methods description would benefit from explicit details on the interview protocol, number of team members interviewed, thematic analysis approach, and any steps taken to mitigate researcher bias in interpreting their own design decisions.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback. We have revised the manuscript to more carefully qualify our claims based on the available data and to better contextualize the scope of the proposed guidelines.

read point-by-point responses
  1. Referee: [Case study evaluation and findings] The central claims attributing sustained teen engagement to gamified/teen-centered features and privacy hurdles to technical instability/rigid structures (abstract and case study findings) rest exclusively on interviews with the research team. This leaves causal attributions vulnerable to self-report bias, as alternative explanations (e.g., twin-specific dynamics or external events) are not ruled out and no direct teen/parent data or raw usage logs are described to corroborate the effects.

    Authors: We recognize the limitation inherent in relying solely on research team interviews for attributing outcomes to design features. This case study was designed to capture the researchers' experiences and interpretations of how the platform influenced the study process. While we cannot rule out alternative explanations without additional data sources, we have revised the text to use more cautious language, such as 'appeared to support' and 'contributed to observed challenges,' rather than implying direct causation. We have also added a dedicated limitations paragraph in the discussion section acknowledging the potential for self-report bias, the specific context of the twin study, and the absence of participant usage logs or direct feedback from teens and parents in this analysis. These revisions aim to strengthen the transparency of our interpretive approach. revision: partial

  2. Referee: [Discussion and guidelines] The leap from this single twin-based case study to general 'actionable interaction design guidelines' for other adolescent EMA contexts (discussion section) overstates the evidence base, given the specific sample characteristics and lack of cross-context validation or falsification attempts.

    Authors: We agree that the guidelines should be presented with appropriate caveats regarding their derivation from a single case study involving adolescent twins in a mental health and sleep study. We have revised the discussion section to explicitly state that these guidelines are informed by this particular context and are intended as starting points for further exploration and validation in diverse adolescent EMA settings. We have removed any phrasing that suggests broad applicability without qualification and included recommendations for future work to test and refine these guidelines across different populations and study designs. revision: yes

Circularity Check

0 steps flagged

No significant circularity; qualitative case study is self-contained

full rationale

This paper is a qualitative case study reporting observations and interview data from a single EMA platform deployment with adolescent twins. It contains no equations, fitted parameters, model-based predictions, or derivation chains. Claims about design features sustaining engagement or creating hurdles rest on direct interview summaries rather than any self-referential reduction or self-citation load-bearing step. The analysis is therefore independent of its inputs by construction and receives the default non-circularity finding.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper rests on standard HCI assumptions about the value of user-centered design and qualitative evaluation; it introduces no new mathematical entities or free parameters.

axioms (1)
  • domain assumption Youth engagement and ethical practice in research can be meaningfully improved by prioritizing teen-centered interface features and researcher support tools.
    Invoked throughout the platform development and guideline extraction.

pith-pipeline@v0.9.0 · 5477 in / 1370 out tokens · 51784 ms · 2026-05-10T15:43:20.315927+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 49 canonical work pages

  1. [1]

    Wisniewski

    Naima Samreen Ali, Renkai Ma, Zainab Agha, Jinkyung Katie Park, and Pamela J. Wisniewski. 2025. Teens as Co-Researchers: Advocating for Disruptive Change to Engage Youth Meaningfully in Online Safety Research and the Design of Social Media.Proc. ACM Hum.-Comput. Interact.9, 7, Article CSCW484 (Oct. 2025), 31 pages. doi:10.1145/3757665

  2. [2]

    30 Days:

    Karla Badillo-Urquiola, Carl Antoine, Amanda Nisenbaum, Zachary Daniel Shea, and Pamela J. Wisniewski. 2022. “30 Days:” An EMA Diary Mobile App & Web Tool. InExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems(New Orleans, LA, USA)(CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 198, 5 pages. do...

  3. [3]

    Karla Badillo-Urquiola, Zachary Shea, Zainab Agha, Irina Lediaeva, and Pamela Wisniewski. 2021. Conducting Risky Research with Teens: Co-designing for the Ethical Treatment and Protection of Adolescents.Proc. ACM Hum.-Comput. Interact.4, CSCW3, Article 231 (Jan. 2021), 46 pages. doi:10.1145/3432930

  4. [4]

    Karla Badillo-Urquiola, Diva Smriti, Brenna McNally, Evan Golub, Elizabeth Bonsignore, and Pamela J Wisniewski. 2019. Stranger danger! social media app features co-designed with children to keep them safe online. InProceedings of the 18th ACM international conference on interaction design and children. 394–406

  5. [5]

    Christopher T Barry, Chloe L Sidoti, Shanelle M Briggs, Shari R Reiter, and Rebecca A Lindsey. 2017. Adolescent social media use and mental health from adolescent and parent perspectives.Journal of adolescence61 (2017), 1–11

  6. [6]

    Caroline Bradbury-Jones, Louise Isham, and Julie Taylor. 2018. The complexities and contradictions in participatory research with vulnerable children and young people: A qualitative systematic review.Soc. Sci. Med.215 (Oct. 2018), 80–91. doi:10.1016/j.socscimed.2018.08.038

  7. [7]

    Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health11, 4 (2019), 589–597

  8. [8]

    Bridgette M Brawner, Ellen M Volpe, Jennifer M Stewart, and Melissa M Gomes

  9. [9]

    Attitudes and beliefs toward biobehavioural research participation: voices and concerns of urban adolescent females receiving outpatient mental health treatment.Annals of human biology40, 6 (2013), 485–495

  10. [10]

    Katharina Buckmayer, Filipa Rocha, Elisa Rubegni, Reem Talhouk, Hugo Nicolau, and Ana Cristina Pires. 2024. Participatory Design with Young Children: Failures, Challenges, and Successes. InProceedings of the 23rd Annual ACM Interaction Design and Children Conference(Delft, Netherlands)(IDC ’24). Association for Computing Machinery, New York, NY, USA, 1038...

  11. [11]

    Xavier V Caddle, Nurun Naher, Zachary P Miller, Karla Badillo-Urquiola, and Pamela J Wisniewski. 2023. Duty to respond: The challenges social service providers face when charged with keeping youth safe online.Proceedings of the ACM on Human-Computer Interaction7, GROUP (2023), 1–35

  12. [12]

    Lucie Cluver, Jenny Doubt, Teen Advisory Groups South Africa, Inge Wessels, Carine Asnong, Samantha Malunga, Kerry Mauchline, Beth Vale, Sally Medley, Elona Toska, et al . 2021. Power to participants: methodological and ethical reflections from a decade of adolescent advisory groups in South Africa.AIDS care33, 7 (2021), 858–866

  13. [13]

    W Scott Comulada, Marguerita Lightfoot, Dallas Swendeman, Christine Grella, and Nancy Wu. 2015. Compliance to cell phone-based EMA among Latino youth in outpatient treatment.Journal of ethnicity in substance abuse14, 3 (2015), 232–250

  14. [14]

    Stacey Crane and Marion E. Broome. 2017. Understanding Ethical Issues of Research Participation from the Perspective of Participating Children and Ado- lescents: A Systematic Review.Worldviews Evid. Based Nurs.14, 3 (Feb. 2017),

  15. [15]

    doi:10.1111/wvn.12209

  16. [16]

    Olivia Cullen and Christine A. Walsh. 2020. A Narrative Review of Ethical Issues in Participatory Research with Young People.YOUNG(Sept. 2020). doi:10.1177/ 1103308819886470

  17. [17]

    Evelien Dietvorst, Manon HJ Hillegers, Jeroen S Legerstee, Lianne P De Vries, Annabel Vreeker, and Loes Keijsers. 2024. Real-time personalized feedback in mHealth for adolescents.Digital Health10 (2024), 20552076241247937

  18. [18]

    Elizabeth Foss, Allison Druin, and Mona Leigh Guha. 2013. Recruiting and retaining young participants: Strategies from five years of field research. In Proceedings of the 12th International Conference on Interaction Design and Children. 313–316

  19. [19]

    Rebecca Grist, Joanna Porter, and Paul Stallard. 2018. Acceptability, use, and safety of a mobile phone app (BlueIce) for young people who self-harm: qualitative study of service users’ experience.JMIR mental health5, 1 (2018), e8779

  20. [20]

    Ole Sejer Iversen and Rachel Charlotte Smith. 2012. Scandinavian participatory design: dialogic curation with teenagers. InProceedings of the 11th International Conference on Interaction Design and Children(Bremen, Germany)(IDC ’12). Association for Computing Machinery, New York, NY, USA, 106–115. doi:10. 1145/2307096.2307109

  21. [21]

    Minal Jain and Pradeep Yammiyavar. 2015. Game based learning tool seeking peer support for empowering adolescent girls in rural Assam. InProceedings of the 14th International Conference on Interaction Design and Children(Boston, Massachusetts)(IDC ’15). Association for Computing Machinery, New York, NY, USA, 275–278. doi:10.1145/2771839.2771895

  22. [22]

    Stephanie T Jong, Rebecca Stevenson, Eleanor M Winpenny, Kirsten Corder, and Esther MF van Sluijs. 2023. Recruitment and retention into longitudinal health research from an adolescent perspective: a qualitative study.BMC Medical Research Methodology23, 1 (2023), 16

  23. [23]

    Clara Rübner Jørgensen and Clara Rübner Jørgensen. 2019. Children’s Involve- ment in Research—A Review and Comparison with Service User Involvement in Health and Social Care.Social Sciences8, 5 (May 2019). doi:10.3390/socsci8050149

  24. [24]

    Soowon Kang, Cheul Young Park, Auk Kim, Narae Cha, and Uichin Lee. 2022. Understanding emotion changes in mobile experience sampling. InProceedings of the 2022 CHI conference on human factors in computing systems. 1–14

  25. [25]

    Matthias F Limberger, Florian Schmiedek, Philip S Santangelo, Markus Reichert, Lena M Wieland, Oksana Berhe, Andreas Meyer-Lindenberg, Heike Tost, and Ulrich W Ebner-Priemer. 2023. Assessing affect in adolescents with e-diaries: multilevel confirmatory factor analyses of different factor models.Frontiers in Psychology14 (2023), 1061229

  26. [26]

    Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice.Proceedings of the ACM on human-computer interaction3, CSCW (2019), 1–23

  27. [27]

    Teresa M McIntyre, Scott E McIntyre, Christopher D Barr, Phillip S Woodward, David J Francis, Angelia C Durand, Paras Mehta, and Thomas W Kamarck. 2016. Longitudinal study of the feasibility of using ecological momentary assessment to study teacher stress: Objective and self-reported measures.Journal of Occupa- tional Health Psychology21, 4 (2016), 403. I...

  28. [28]

    Abebe Walle Menberu. 2024. Technology-mediated financial education in devel- oping countries: A systematic literature review.Cogent Business & Management 11, 1 (2024), 2294879

  29. [29]

    Michelle MJ Mens, Loes Keijsers, Evelien Dietvorst, Soldado Koval, Jeroen S Leg- erstee, and Manon HJ Hillegers. 2022. Promoting daily well-being in adolescents using mHealth.Journal of Youth and Adolescence51, 11 (2022), 2173–2189

  30. [30]

    Aja Louise Murray and Tong Xie. 2024. Engaging adolescents in contemporary longitudinal health research: Strategies for promoting participation and retention. Journal of Adolescent Health74, 1 (2024), 9–17

  31. [31]

    It helped me feel like a researcher

    Kendra Nelson Ferguson, Stephanie E Coen, and Jason Gilliland. 2025. “It helped me feel like a researcher”: reflections on a capacity-building program to support teens as co-researchers on a participatory project.Journal of adolescent research 40, 4 (2025), 827–858

  32. [32]

    Oguine, Adriana Alvarado Garcia, Michael Muller, and Karla Badillo- Urquiola

    Ozioma C. Oguine, Adriana Alvarado Garcia, Michael Muller, and Karla Badillo- Urquiola. 2026. Who Gets to Define Safety? A Systematic Review of How Gener- ative AI Research Addresses Youth Online Safety.ResearchGate(March 2026). doi:10.1145/3772318.3791346

  33. [33]

    Oguine, Jinkyung Katie Park, Mamtaj Akter, Johanna Olesk, Abdul- malik Alluhidan, Pamela Wisniewski, and Karla Badillo-Urquiola

    Ozioma C. Oguine, Jinkyung Katie Park, Mamtaj Akter, Johanna Olesk, Abdul- malik Alluhidan, Pamela Wisniewski, and Karla Badillo-Urquiola. 2025. How the Internet Facilitates Adverse Childhood Experiences for Youth Who Self-Identify as in Need of Services.Proc. ACM Hum.-Comput. Interact.9, 2, Article CSCW097 (May 2025), 39 pages. doi:10.1145/3710995

  34. [34]

    Oguine, Danielle Radford, Samantha Hoang, and Karla Badillo- Urquiola

    Ozioma C. Oguine, Danielle Radford, Samantha Hoang, and Karla Badillo- Urquiola. 2025. Inclusion by Design or by Default? Examining How HCI Studies Engage Underrepresented Youth in Makerspaces. InCompanion Publication of the 2025 Conference on Computer-Supported Cooperative Work and Social Computing. 326–334

  35. [35]

    Noh, Deborah A

    Jinkyung Katie Park, Renkai Ma, Naima Samreen Ali, Naulsberry Jean Baptiste, Zainab Agha, and Pamela J. Wisniewski. 2025. Teens, Privacy, and Algorithms: Navigating and Co-Designing Solutions for Interpersonal Boundary Management on Social Media. InProceedings of the 24th Interaction Design and Children (IDC ’25). Association for Computing Machinery, New ...

  36. [36]

    Erika S Poole and Tamara Peyton. 2013. Interaction design research with adoles- cents: methodological challenges and best practices. InProceedings of the 12th international conference on interaction design and children. 211–217

  37. [37]

    Stones, Hugh Davies, Jennifer Preston, and Bob Phillips

    Jenny Preston, Simon R. Stones, Hugh Davies, Jennifer Preston, and Bob Phillips

  38. [38]

    How to involve children and young people in what is, after all, their research.Arch. Dis. Child.104, 5 (May 2019), 494–500. arXiv:31000534 doi:10. 1136/archdischild-2018-315118

  39. [39]

    Arizona Twin Project. 2023. Arizona Twin Project. https://arizonatwinproject. org/research

  40. [40]

    Galambos, Andrea L

    Spring C. Cooper Robbins, Margot Rawsthorne, Karen Paxton, Catherine Hawke, S. Rachel Skinner, and Katharine Steinbeck. 2011. “You Can Help People”: Ado- lescents’ Views on Engaging Young People in Longitudinal Research.Jour- nal of Research on Adolescence22, 1 (Sept. 2011), 8–13. doi:10.1111/j.1532- 7795.2011.00759.x

  41. [41]

    Elise Sellars, Gabriela Pavarini, Daniel Michelson, Cathy Creswell, and Mina Fazel

  42. [42]

    Young people’s advisory groups in health research: scoping review and mapping of practices.Arch. Dis. Child.106, 7 (July 2021), 698–704. arXiv:33208398 doi:10.1136/archdischild-2020-320452

  43. [43]

    Vishal Sharma, Shivani A Mehta, Neha Kumar, and Aaditeshwar Seth. 2024. Whose Participation Counts? Towards Technology-Mediated Equitable Futures of Development Work.Proceedings of the ACM on Human-Computer Interaction 8, CSCW2 (2024), 1–30

  44. [44]

    Mariya P Shiyko, Seth Perkins, and Linda Caldwell. 2017. Feasibility and adher- ence paradigm to ecological momentary assessments in urban minority youth. Psychological Assessment29, 7 (2017), 926

  45. [45]

    Ge Wang, Jun Zhao, Max Van Kleek, and Nigel Shadbolt. 2023. 12 Ways to empower: Designing for children’s digital autonomy. InProceedings of the 2023 CHI conference on human factors in computing systems. 1–27

  46. [46]

    Azza Warraitch, Ciara Wacker, Sanjana Biju, Maria Lee, Delali Bruce, Paul Curran, Qusai Khraisha, and Kristin Hadfield. 2024. Positive Impacts of Adolescent Involvement in Health Research: An Umbrella Review.J. Adolesc. Health75, 2 (Aug. 2024), 218–230. arXiv:38597838 doi:10.1016/j.jadohealth.2024.02.029

  47. [47]

    Azza Warraitch, Ciara Wacker, Emer Buckley, Ashling Bourke, and Kristin Had- field. 2024. How to address the barriers to meaningful adolescent involvement in health research: A qualitative study.J. Res. Adolesc.34, 4 (Dec. 2024), 1626–1641. arXiv:39428699 doi:10.1111/jora.13031

  48. [48]

    CA Webb, CM Swords, L Murray, and LM Hilt. 2021. App-based mindfulness training for adolescent rumination: Predictors of immediate and cumulative benefit. Mindfulness, 12 (10), 2498–2509

  49. [49]

    O Wilson, L Daxenberger, L Dieudonne, J Eustace, A Hanard, A Krishnamurthi, P Quigley, and A Vergou. 2020. A rapid evidence review of young people’s involvement in health research.London: Wellcome3 (2020)