pith. sign in

arxiv: 1906.10428 · v1 · pith:3ESEEWAOnew · submitted 2019-06-25 · 💻 cs.HC · cs.MM

A novel music-based game with motion capture to support cognitive and motor function in the elderly

Kat Agres , Simon Lui , Dorien Herremans This is my paper

Pith reviewed 2026-05-25 16:35 UTC · model grok-4.3

classification 💻 cs.HC cs.MM
keywords music-based gamemotion captureelderly carecognitive functionmotor functionKinecteHealthpreventive medicine
0
0 comments X

The pith

A music-based game prototype uses memory tasks and Kinect-detected instrument gestures to support cognitive and motor function in the elderly.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a game prototype that combines music with motion detection to act as preventive medicine for the elderly. It employs a memory task to stimulate cognitive function and requires physical gestures mimicking musical instruments, detected via the Microsoft Kinect sensor and a custom gesture module. The system aims to motivate users who often have low compliance with physical exercises by leveraging music's known benefits for mental and physical wellness. This addresses the challenges of aging populations and limited care resources through eHealth solutions.

Core claim

The resulting prototype system supports both cognitive functioning and physical strengthening in the elderly by using music to engage users in a memory task while they perform instrument-playing gestures detected by motion capture.

What carries the argument

A memory task combined with instrument-mimicking gestures detected by the Microsoft Kinect sensor and a new gesture detection module.

Load-bearing premise

The specific combination of memory tasks and Kinect-detected instrument gestures will lead to measurable benefits in cognitive and motor function.

What would settle it

A controlled study comparing cognitive test scores and physical strength measures before and after regular use of the game versus a control group without the intervention.

Figures

Figures reproduced from arXiv: 1906.10428 by Dorien Herremans, Kat Agres, Simon Lui.

Figure 1
Figure 1. Figure 1: Screenshot of the prototype system and motor support in the elderly that integrates the therapeutic aspects of music. Music is a complex domain that activates many cognitive resources and has shown to have a positive influence on mental health (e.g., by reducing depression and anxiety [10]). In addition, the field of neurological music therapy (NRT) has drawn upon aspects of music perception, such as audit… view at source ↗
read the original abstract

This paper presents a novel game prototype that uses music and motion detection as preventive medicine for the elderly. Given the aging populations around the globe, and the limited resources and staff able to care for these populations, eHealth solutions are becoming increasingly important, if not crucial, additions to modern healthcare and preventive medicine. Furthermore, because compliance rates for performing physical exercises are often quite low in the elderly, systems able to motivate and engage this population are a necessity. Our prototype uses music not only to engage listeners, but also to leverage the efficacy of music to improve mental and physical wellness. The game is based on a memory task to stimulate cognitive function, and requires users to perform physical gestures to mimic the playing of different musical instruments. To this end, the Microsoft Kinect sensor is used together with a newly developed gesture detection module in order to process users' gestures. The resulting prototype system supports both cognitive functioning and physical strengthening in the elderly.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper presents a prototype music-based game that uses Microsoft Kinect for motion capture to detect gestures mimicking musical instruments as part of a memory task, intended to support cognitive and motor function in the elderly as preventive medicine.

Significance. The approach combines music therapy with physical activity in an engaging format, which could be significant for addressing low compliance with exercises in elderly populations if empirical evidence were provided. The current manuscript limits its contribution to a system description without validation.

major comments (1)
  1. [Abstract] Abstract: The assertion that 'the resulting prototype system supports both cognitive functioning and physical strengthening in the elderly' is not backed by any data; the manuscript describes the system design and motivation but includes no user study, performance metrics, error analysis, or statistical results to substantiate the benefits.
minor comments (1)
  1. The distinction between design intentions and demonstrated outcomes could be clarified throughout the text to avoid overstating the prototype's effects.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the detailed review. The manuscript is a system description of a Kinect-based music memory game prototype, and we acknowledge the lack of empirical validation. We address the comment below and will revise accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The assertion that 'the resulting prototype system supports both cognitive functioning and physical strengthening in the elderly' is not backed by any data; the manuscript describes the system design and motivation but includes no user study, performance metrics, error analysis, or statistical results to substantiate the benefits.

    Authors: We agree that the manuscript provides no user study or performance data to substantiate the benefits. This paper focuses on the design, motivation, and implementation of the prototype rather than its evaluation. We will revise the abstract to state that the system is 'designed to support' or 'intended to support' cognitive functioning and physical strengthening, based on the established benefits of music therapy and physical activity cited in the introduction. This change will align the claims with the scope of a system description paper. revision: yes

Circularity Check

0 steps flagged

No derivation chain or self-referential reductions present

full rationale

The manuscript describes a Kinect-based music game prototype for elderly users, with a memory task and instrument gestures. The central claim that the system 'supports both cognitive functioning and physical strengthening' is presented as a design outcome rather than the endpoint of any equations, parameter fitting, or predictive derivation. No self-citations, ansatzes, uniqueness theorems, or fitted inputs appear in the provided text. The paper contains no mathematical steps that could reduce to their own inputs by construction. This is a system-description paper without a load-bearing derivation chain, so the circularity criteria do not apply.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No mathematical model, free parameters, or new entities are introduced; the contribution is a high-level system design description.

pith-pipeline@v0.9.0 · 5694 in / 977 out tokens · 29418 ms · 2026-05-25T16:35:13.670071+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 39 canonical work pages

  1. [1]

    Agres and D

    K. Agres and D. Herremans. 2017. Music and Motion-Detection: A Game Prototype for Rehabilitation and Strengthening in the Elderly. Proc. of IEEE Int. Conference on Orange Technologies (ICOT)

    work page 2017

  2. [2]

    Anderson-Hanley, P.J

    C. Anderson-Hanley, P.J. Arciero, A.M. Brickman, J.P. Nimon, N. Okuma, S.C Westen, M.E. Merz, B.D. Pence, J.A. Woods, and A.F. Kramer. 2012. Exergaming and older adult cognition: a cluster random- ized clinical trial. Am. J. Prev. Med. 42, 2 (2012), 109–119

    work page 2012

  3. [3]

    Ballesteros, A

    S. Ballesteros, A. Prieto, J. Mayas, P. Toril, C. Pita, L. Ponce de Le ´on, J.M. Reales, and J. Waterworth. 2014. Brain training with non-action video games enhances aspects of cognition in older adults: a randomized controlled trial. Frontiers in aging neuroscience 6 (2014), 277

    work page 2014

  4. [4]

    Basak, W.R

    C. Basak, W.R. Boot, M.W. V oss, and A.F. Kramer. 2008. Can training in a real-time strategy video game attenuate cognitive decline in older adults? Psychol. aging 23, 4 (2008), 765

    work page 2008

  5. [5]

    Scott Beveridge, Estefanıa Cano, and Kat Agres. 2018. Rhythmic entrainment for hand rehabilitation using the leap motion controller. 19th International Society for Music Information Retrieval Conference

    work page 2018

  6. [6]

    Brand and R.T

    F.N. Brand and R.T. Smith. 1974. Medical care and compliance among the elderly after hospitalization. Int. J. Aging Hum. Dev. 5, 4 (1974), 331–346

    work page 1974

  7. [7]

    W.-C. Chan. 2011. Singapore’s Ageing Population: Managing Health- care and End-of-life Decisions . V ol. 34. Routledge

    work page 2011

  8. [8]

    Chang, S.-F

    Y .-J. Chang, S.-F. Chen, and J.-D. Huang. 2011. A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Res. Dev. Disabil. 32, 6 (2011), 2566–2570

    work page 2011

  9. [9]

    Yao-Jen Chang, Wen-Ying Han, and Yu-Chi Tsai. 2013. A Kinect-based upper limb rehabilitation system to assist people with cerebral palsy. Research in developmental disabilities 34, 11 (2013), 3654–3659

    work page 2013

  10. [10]

    Clair and J

    A.A. Clair and J. Memmott. 2008. Therapeutic uses of music with older adults. American Music Therapy Association

    work page 2008

  11. [11]

    Clark, Y .-H

    R.A. Clark, Y .-H. Pua, K. Fortin, C. Ritchie, K.E. Webster, L. Denehy, and A.L. Bryant. 2012. Validity of the Microsoft Kinect for assessment of postural control. Gait & posture 36, 3 (2012), 372–377

    work page 2012

  12. [12]

    Ellis, Y

    R.J. Ellis, Y . S. Ng, S. Zhu, .M. Tan, B. Anderson, G. Schlaug, and Y . Wang. 2015. A validated smartphone-based assessment of gait and gait variability in Parkinson’s disease. PLoS one 10, 10 (2015), e0141694

    work page 2015

  13. [13]

    K.E. Gfeller. 1983. Musical mnemonics as an aid to retention with normal and learning disabled students. Journal of Music Therapy 20, 4 (1983), 179–189

    work page 1983

  14. [14]

    Guest and M.D

    A. Guest and M.D. Apgar. 2002. Promoting and prescribing exercise for the elderly. Americ. family physician 65 (2002), 3

    work page 2002

  15. [15]

    Hanna-Pladdy and A

    B. Hanna-Pladdy and A. MacKay. 2011. The relation between instru- mental musical activity and cognitive aging. Neuropsychology 25, 3 (2011), 378

    work page 2011

  16. [16]

    Irish, C.J

    M. Irish, C.J. Cunningham, J.B. Walsh, D. Coakley, B.A. Lawlor, I.H. Robertson, and R.F. Coen. 2006. Investigating the enhancing effect of music on autobiographical memory in mild Alzheimer’s disease. Dement. Geriatr. Cogn. 22, 1 (2006), 108–120

    work page 2006

  17. [17]

    Kayama, K

    H. Kayama, K. Okamoto, S. Nishiguchi, M. Yamada, T. Kuroda, and T. Aoyama. 2014. Effect of a Kinect-based exercise game on improving executive cognitive performance in community-dwelling elderly: case control study. Journal of medical Internet research 16, 2 (2014)

    work page 2014

  18. [18]

    Khalfa, S.D

    S. Khalfa, S.D. Bella, M. Roy, I. Peretz, and S.J. Lupien. 2003. Effects of relaxing music on salivary cortisol level after psychological stress. Ann. N. Y. Acad. Sci. 999, 1 (2003), 374–376

    work page 2003

  19. [19]

    Kharrazi, Amy S

    H. Kharrazi, Amy S. Lu, F. Gharghabi, and W. Coleman. 2012. A scop- ing review of health game research: Past, present, and future. GAMES FOR HEALTH: Research, Development, and Clinical Applications 1, 2 (2012), 153–164

    work page 2012

  20. [20]

    J. Kim, T. Wigram, and C. Gold. 2008. The effects of improvisational music therapy on joint attention behaviors in autistic children: a random- ized controlled study. Journal of autism and developmental disorders 38, 9 (2008), 1758

    work page 2008

  21. [21]

    Kwok, K.C

    T.C.Y . Kwok, K.C. Lam, P.S. Wong, W.W. Chau, K.S.L. Yuen, K.T. Ting, E.W.K. Chung, J.C.Y . Li, and F.K.Y . Ho. 2011. Effectiveness of coordination exercise in improving cognitive function in older adults: a prospective study. Clinical interventions in aging 6 (2011), 261

    work page 2011

  22. [22]

    Lai, C.-W

    C.-H. Lai, C.-W. Peng, Y .-L. Chen, C.-P. Huang, Y .-L. Hsiao, and S.-C. Chen. 2013. Effects of interactive video-game based system exercise on the balance of the elderly. Gait & posture 37, 4 (2013), 511–515

    work page 2013

  23. [23]

    Lange, C.-Y

    B. Lange, C.-Y . Chang, E. Suma, B. Newman, A.S. Rizzo, and M. Bolas

    work page

  24. [24]

    In Engineering in medicine and biology society, EMBC, 2011 Ann

    Development and evaluation of low cost game-based balance rehabilitation tool using the Microsoft Kinect sensor. In Engineering in medicine and biology society, EMBC, 2011 Ann. Int. Conf. of the IEEE . IEEE, 1831–1834

    work page 2011

  25. [25]

    Laurin, R

    D. Laurin, R. Verreault, J. Lindsay, K. MacPherson, and K. Rockwood

    work page

  26. [26]

    Archives of neurology 58, 3 (2001), 498–504

    Physical activity and risk of cognitive impairment and dementia in elderly persons. Archives of neurology 58, 3 (2001), 498–504

    work page 2001

  27. [27]

    Nasreddine, N.A

    Z.S. Nasreddine, N.A. Phillips, V . B ´edirian, S. Charbonneau, V . White- head, I. Collin, J.L. Cummings, and H. Chertkow. 2005. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 53, 4 (2005), 695–699

    work page 2005

  28. [28]

    United Nations. 2017. World ageing population - Highlights . Technical Report. Department of Economic and Social Affairs, NY

    work page 2017

  29. [29]

    Ngandu, J

    T. Ngandu, J. Lehtisalo, A. Solomon, E. Lev ¨alahti, S. Ahtiluoto, R. Antikainen, L. B¨ackman, T. H¨anninen, A. Jula, and T. Laatikainen. 2015. A 2 year multidomain intervention of diet, exercise, cognitive training, and vascular risk monitoring versus control to prevent cognitive decline in at-risk elderly people (FINGER): a randomised controlled trial. ...

    work page 2015

  30. [30]

    Prickett and R.S

    C.A. Prickett and R.S. Moore. 1991. The use of music to aid memory of Alzheimer’s patients. Journal of Music Therapy 28, 2 (1991), 101–110

    work page 1991

  31. [31]

    Simmons-Stern, A.E

    N.R. Simmons-Stern, A.E. Budson, and B.A. Ally. 2010. Music as a memory enhancer in patients with Alzheimer’s disease. Neuropsycholo- gia 48, 10 (2010), 3164–3167

    work page 2010

  32. [32]

    M.H. Thaut. 2005. Rhythm, music, and the brain: Scientific foundations and clinical applications . V ol. 7. Routledge

    work page 2005

  33. [33]

    Thaut, J.C

    M.H. Thaut, J.C. Gardiner, D. Holmberg, J. Horwitz, L. Kent, G. An- drews, B. Donelan, and G.R. McIntosh. 2009. Neurologic music therapy improves executive function and emotional adjustment in traumatic brain injury rehabilitation. Ann. N. Y. Acad. Sci. 1169, 1 (2009), 406–416

    work page 2009

  34. [34]

    Tombaugh and N.J

    T.N. Tombaugh and N.J. McIntyre. 1992. The mini-mental state examination: a comprehensive review. J. Am. Geriatr. Soc. 40, 9 (1992), 922–935

    work page 1992

  35. [35]

    Toril, J.M

    P. Toril, J.M. Reales, and S. Ballesteros. 2014. Video game training enhances cognition of older adults: a meta-analytic study. Psychol. and aging 29, 3 (2014), 706

    work page 2014

  36. [36]

    Washburn, K.W

    R.A. Washburn, K.W. Smith, A.M. Jette, and C.A. Janney. 1993. The Physical Activity Scale for the Elderly (PASE): development and evaluation. Journal of clinical epidemiology 46, 2 (1993), 153–162

    work page 1993

  37. [37]

    J.M. White. 1992. Music therapy: an intervention to reduce anxiety in the myocardial infarction patient. Clin. Nurse Spec. 6, 2 (1992), 58–63

    work page 1992

  38. [38]

    Wolfe and C

    D.E. Wolfe and C. Hom. 1993. Use of melodies as structural prompts for learning and retention of sequential verbal information by preschool students. Journal of Music therapy 30, 2 (1993), 100–118

    work page 1993

  39. [39]

    Z. Zhang. 2012. Microsoft kinect sensor and its effect. IEEE multimed. 19, 2 (2012), 4–10

    work page 2012