EVA: Generating Emotional Behavior of Virtual Agents using Expressive Features of Gait and Gaze
Pith reviewed 2026-05-25 09:38 UTC · model grok-4.3
The pith
The EVA algorithm generates emotional virtual agents from gait and gaze features, increasing sense of presence in multi-agent VR.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
EVA is a novel real-time algorithm that uses a precomputed data-driven mapping between gaits and perceived emotions to select appropriate walking styles and gazing behaviors, thereby generating virtual agents that convey happy, sad, angry, or neutral states. The approach enables simulation of gaits and gazing behaviors for hundreds of agents simultaneously. Evaluations across different multi-agent VR simulation environments indicate that these expressive features considerably increase the sense of presence.
What carries the argument
The precomputed data-driven mapping between gaits and their perceived emotions, applied at runtime to drive both walking styles and gaze for emotional expression.
If this is right
- Hundreds of virtual agents can be simulated in real time while maintaining known emotional characteristics.
- Sense of presence rises in VR scenarios that contain multiple agents.
- The same expressive features apply across different multi-agent VR simulation environments.
- Four discrete emotion categories can be conveyed through the gait-gaze combination.
Where Pith is reading between the lines
- If the mapping generalizes, designers could adjust the emotional mix of a virtual crowd to tune immersion levels without changing geometry or lighting.
- Similar precomputed mappings might be built for other actions such as gesturing or sitting to extend emotional control beyond locomotion.
- The presence benefit could be tested in larger or interactive crowds to check whether added complexity preserves or amplifies the effect.
Load-bearing premise
The precomputed mapping from gaits to perceived emotions remains accurate and consistent when applied to new observers, new VR scenes, and varying numbers of agents.
What would settle it
A controlled VR study measuring presence scores for scenes populated with EVA emotional agents versus identical scenes using neutral gaits and gazes, with no statistically significant difference between the two conditions.
Figures
read the original abstract
We present a novel, real-time algorithm, EVA, for generating virtual agents with various perceived emotions. Our approach is based on using Expressive Features of gaze and gait to convey emotions corresponding to happy, sad, angry, or neutral. We precompute a data-driven mapping between gaits and their perceived emotions. EVA uses this gait emotion association at runtime to generate appropriate walking styles in terms of gaits and gaze. Using the EVA algorithm, we can simulate gaits and gazing behaviors of hundreds of virtual agents in real-time with known emotional characteristics. We have evaluated the benefits in different multi-agent VR simulation environments. Our studies suggest that the use of expressive features corresponding to gait and gaze can considerably increase the sense of presence in scenarios with multiple virtual agents.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents EVA, a real-time algorithm for generating emotional behaviors (happy, sad, angry, neutral) in virtual agents via expressive gait and gaze features. It precomputes a data-driven mapping from gaits to perceived emotions and applies the mapping at runtime to control walking styles and gaze for hundreds of agents. Evaluations in multi-agent VR environments indicate that these expressive features considerably increase users' sense of presence.
Significance. If the gait-emotion mapping proves accurate, consistent, and generalizable, the work offers a practical method for real-time emotional crowd simulation in VR, with potential to improve immersion in scenarios involving multiple agents. The emphasis on real-time performance for large agent counts is a practical strength for interactive applications.
major comments (1)
- The central claim that expressive gait and gaze features increase presence rests on the accuracy and generalizability of the precomputed gait-emotion mapping, yet the manuscript provides no details on the mapping study's design (participant count, stimuli, inter-rater reliability, or cross-validation), preventing assessment of whether presence gains can be attributed to the features rather than other variables.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback. We address the single major comment below and agree that additional details are needed.
read point-by-point responses
-
Referee: The central claim that expressive gait and gaze features increase presence rests on the accuracy and generalizability of the precomputed gait-emotion mapping, yet the manuscript provides no details on the mapping study's design (participant count, stimuli, inter-rater reliability, or cross-validation), preventing assessment of whether presence gains can be attributed to the features rather than other variables.
Authors: We agree with the referee that the manuscript as submitted does not include sufficient methodological details on the user study used to derive the gait-emotion mapping. This information is necessary to evaluate the mapping's reliability and to support the attribution of presence improvements to the expressive features. We will revise the paper by adding a dedicated subsection (likely in Section 3 or 4) that reports the participant count, stimuli, inter-rater reliability, and cross-validation procedures from the original mapping study. This addition will directly address the concern and strengthen the evidential basis for the central claims. revision: yes
Circularity Check
No significant circularity in derivation chain
full rationale
The paper precomputes a data-driven mapping from observed gaits to perceived emotions (happy/sad/angry/neutral) via an external study, then applies this fixed mapping at runtime to control gait and gaze parameters for virtual agents. The central claim—that expressive gait+gaze features increase presence—is tested via separate user studies in multi-agent VR environments. No equations, self-citations, or ansatzes are presented that reduce the runtime outputs or presence ratings back to the mapping inputs by construction; the mapping itself is treated as an empirical input rather than a fitted prediction of the target metric. The derivation is therefore self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
free parameters (1)
- gait-emotion mapping parameters
axioms (1)
- domain assumption Human observers consistently perceive emotions from gait and gaze features in a manner that can be captured by data-driven mapping.
Reference graph
Works this paper leans on
-
[1]
CMU Graphics Lab Motion Capture Database
2018. CMU Graphics Lab Motion Capture Database. http://mocap.cs.cmu.edu/
work page 2018
-
[2]
Retargeting of Humanoid animations
2019. Retargeting of Humanoid animations. https://docs.unity3d.com/Manual/ Retargeting. Accessed: 2019-06-30
work page 2019
-
[3]
Reginald B Adams and Robert E Kleck. 2003. Perceived gaze direction and the processing of facial displays of emotion. Psychological Science 14, 6 (2003), 644–647
work page 2003
-
[4]
Norman Badler, Jan Allbeck, Liwei Zhao, and Meeran Byun. 2002. Representing and parameterizing agent behaviors. In Computer Animation, 2002. Proceedings of. IEEE, 133–143
work page 2002
-
[5]
Jeremy N Bailenson, Andrew C Beall, Jack Loomis, Jim Blascovich, and Matthew Turk. 2005. Transformed social interaction, augmented gaze, and social influence in immersive virtual environments. Human communication research 31, 4 (2005), 511–537
work page 2005
-
[6]
L. F. Barrett. 2017. How emotions are made: The secret life of the brain . Houghton Mifflin Harcourt
work page 2017
-
[7]
Jenay M Beer, Arthur D Fisk, and Wendy A Rogers. 2009. Emotion recognition of virtual agents facial expressions: the effects of age and emotion intensity. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting , Vol. 53. SAGE Publications Sage CA: Los Angeles, CA, 131–135
work page 2009
-
[8]
Aniket Bera, Tanmay Randhavane, Emily Kubin, Husam Shaik, Kurt Gray, and Dinesh Manocha. 2018. Data-driven modeling of group entitativity in virtual environments. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology. ACM, 31
work page 2018
-
[9]
Aniket Bera, Tanmay Randhavane, and Dinesh Manocha. 2017. Aggressive, Tense or Shy? Identifying Personality Traits from Crowd Videos.. In IJCAI. 112–118
work page 2017
-
[10]
D Bernhardt and P Robinson. 2007. Detecting affect from non-stylised body motions. In ACII
work page 2007
-
[11]
Andry Chowanda, Peter Blanchfield, Martin Flintham, and Michel Valstar. 2016. Computational Models of Emotion, Personality, and Social Relationships for Interactions in Games: (Extended Abstract). In AAMAS
work page 2016
-
[12]
Céline Clavel, Justine Plessier, Jean-Claude Martin, Laurent Ach, and Benoit Morel. 2009. Combining facial and postural expressions of emotions in a virtual character. In International Workshop on Intelligent Virtual Agents . Springer, 287– 300
work page 2009
-
[13]
Arthur Crenn, Rizwan Ahmed Khan, Alexandre Meyer, and Saida Bouakaz. 2016. Body expression recognition from animated 3D skeleton. In 2016 International Conference on 3D Imaging (IC3D) . IEEE, 1–7
work page 2016
-
[14]
Arthur Crenn, Alexandre Meyer, Rizwan Ahmed Khan, Hubert Konik, and Saïda Bouakaz. 2017. Toward an efficient body expression recognition based on the synthesis of a neutral movement. In Proceedings of the 19th ACM International Conference on Multimodal Interaction . ACM, 15–22
work page 2017
-
[15]
Rishabh Dabral, Anurag Mundhada, Uday Kusupati, Safeer Afaque, Abhishek Sharma, and Arjun Jain. 2018. Learning 3d human pose from structure and motion. In Proceedings of the European Conference on Computer Vision (ECCV) . 668–683
work page 2018
-
[16]
Mohamed Daoudi, Stefano Berretti, Pietro Pala, Yvonne Delevoye, and Alberto Del Bimbo. 2017. Emotion recognition by body movement representation on the manifold of symmetric positive definite matrices. In International Conference on Image Analysis and Processing. Springer, 550–560
work page 2017
-
[17]
Funda Durupinar, Mubbasir Kapadia, Susan Deutsch, Michael Neff, and Norman I Badler. 2017. Perform: Perceptual approach for adding ocean personality to human motion using laban movement analysis. ACM Transactions on Graphics (TOG) 36, 1 (2017), 6. Symposium on Applied Perception, September 2019, Barcelona, Spain Tanmay Randhavane, Aniket Bera, Kyra Kapsas...
work page 2017
-
[18]
Paul Ekman and Wallace V Friesen. 1967. Head and body cues in the judgment of emotion: A reformulation. Perceptual and motor skills 24, 3 PT 1 (1967), 711–724
work page 1967
-
[19]
Ylva Ferstl and Rachel McDonnell. 2018. A perceptual study on the manipulation of facial features for trait portrayal in virtual agents. In Proceedings of the 18th International Conference on Intelligent Virtual Agents . ACM, 281–288
work page 2018
-
[20]
Andrew C Gallup, Andrew Chong, Alex Kacelnik, John R Krebs, and Iain D Couzin. 2014. The influence of emotional facial expressions on gaze-following in grouped and solitary pedestrians. Scientific reports 4 (2014), 5794
work page 2014
-
[21]
Maia Garau, Mel Slater, David-Paul Pertaub, and Sharif Razzaque. 2005. The responses of people to virtual humans in an immersive virtual environment. Presence: Teleoperators & Virtual Environments 14, 1 (2005), 104–116
work page 2005
-
[22]
Beatrice de Gelder. 2016. Emotions and the body . Oxford University Press
work page 2016
-
[23]
Tom Geller. 2008. Overcoming the uncanny valley. IEEE computer graphics and applications 28, 4 (2008), 11–17
work page 2008
-
[24]
Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. 2013. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelligence 36, 7 (2013), 1325–1339
work page 2013
-
[25]
Natasha Jaques, Daniel McDuff, Yoo Lim Kim, and Rosalind Picard. 2016. Un- derstanding and predicting bonding in conversations using thin slices of facial expressions and body language. In International Conference on Intelligent Virtual Agents. Springer, 64–74
work page 2016
-
[26]
Michelle Karg, Kolja Kuhnlenz, and Martin Buss. 2010. Recognition of affect based on gait patterns. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 40, 4 (2010), 1050–1061
work page 2010
-
[27]
A. Kleinsmith and N. Bianchi-Berthouze. 2013. Affective Body Expression Per- ception and Recognition: A Survey. IEEE Transactions on Affective Computing 4, 1 (Jan 2013), 15–33. https://doi.org/10.1109/T-AFFC.2012.16
-
[28]
Andrea Kleinsmith, P Ravindra De Silva, and Nadia Bianchi-Berthouze. 2005. Grounding affective dimensions into posture features. InInternational Conference on Affective Computing and Intelligent Interaction . Springer, 263–270
work page 2005
-
[29]
Mariska Esther Kret, Karin Roelofs, Jeroen Stekelenburg, and Beatrice de Gelder
-
[30]
Frontiers in human neuroscience 7 (2013), 810
Emotional signals from faces, bodies and scenes influence observers’ face expressions, fixations and pupil-size. Frontiers in human neuroscience 7 (2013), 810
work page 2013
-
[31]
Brent J Lance and Stacy C Marsella. 2008. A model of gaze for the purpose of emotional expression in virtual embodied agents. In Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems- Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 199–206
work page 2008
-
[32]
Benny Ping-Han Lee, Edward Chao-Chun Kao, and Von-Wun Soo. 2006. Feel- ing ambivalent: A model of mixed emotions for virtual agents. In International Workshop on Intelligent Virtual Agents. Springer, 329–342
work page 2006
-
[33]
Benny Liebold. 2016. Cognitive and Emotional Processing of Virtual Environments: the Significance of Attentional Processes and Mental Models . Ph.D. Dissertation. Technische Universität Chemnitz
work page 2016
-
[34]
Benny Liebold and Peter Ohler. 2013. Multimodal emotion expressions of virtual agents, mimic and vocal emotion expressions and their effects on emotion recog- nition. In Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on. IEEE, 405–410
work page 2013
-
[35]
Zhen Liu, Tingting Liu, Minhua Ma, Hui-Huang Hsu, Zhongrui Ni, and Yanjie Chai. 2018. A perception-based emotion contagion model in crowd emergent evacuation simulation. Computer Animation and Virtual Worlds 29, 3-4 (2018), e1817
work page 2018
-
[36]
Zhen Liu and Zhi Geng Pan. 2005. An emotion model of 3d virtual charac- ters in intelligent virtual environment. In International Conference on Affective Computing and Intelligent Interaction . Springer, 629–636
work page 2005
-
[37]
Matthew Lombard, Theresa B Ditton, and Lisa Weinstein. 2009. Measuring presence: the temple presence inventory. In Proceedings of the 12th Annual Inter- national Workshop on Presence. 1–15
work page 2009
-
[38]
Sebastian Loth, Gernot Horstmann, Corinnna Osterbrink, and Stefan Kopp. 2018. Accuracy of Perceiving Precisely Gazing Virtual Agents. In Proceedings of the 18th International Conference on Intelligent Virtual Agents . ACM, 263–268
work page 2018
-
[39]
Yingliang Ma, Helena M Paterson, and Frank E Pollick. 2006. A motion capture library for the study of identity, gender, and emotion perception from biological motion. Behavior research methods 38, 1 (2006), 134–141
work page 2006
-
[40]
Joanna Edel McHugh, Rachel McDonnell, Carol OâĂŹSullivan, and Fiona N Newell. 2010. Perceiving emotion in crowds: the role of dynamic body postures on the perception of emotion in crowded scenes. Experimental brain research 204, 3 (2010), 361–372
work page 2010
-
[41]
Albert Mehrabian. 1980. Basic dimensions for a general psychological theory implications for personality, social, environmental, and developmental studies. (1980)
work page 1980
-
[42]
Albert Mehrabian. 1996. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology 14, 4 (1996), 261–292
work page 1996
-
[43]
Joseph A Mikels, Barbara L Fredrickson, Gregory R Larkin, Casey M Lindberg, Sam J Maglio, and Patricia A Reuter-Lorenz. 2005. Emotional category data on images from the International Affective Picture System. Behavior research methods 37, 4 (2005), 626–630
work page 2005
-
[44]
Jon D Morris. 1995. Observations: SAM: the Self-Assessment Manikin; an efficient cross-cultural measurement of emotional response.Journal of advertising research 35, 6 (1995), 63–68
work page 1995
-
[45]
Sahil Narang, Andrew Best, Andrew Feng, Sin-hwa Kang, Dinesh Manocha, and Ari Shapiro. 2017. Motion recognition of self and others on realistic 3D avatars. Computer Animation and Virtual Worlds 28, 3-4 (2017), e1762
work page 2017
-
[46]
Sahil Narang, Andrew Best, Tanmay Randhavane, Ari Shapiro, and Dinesh Manocha. 2016. PedVR: Simulating gaze-based interactions between a real user and virtual crowds. In Proceedings of the 22nd ACM conference on virtual reality software and technology. ACM, 91–100
work page 2016
-
[47]
R. S. Nickerson. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. Review of general psychology 2, 2 (1998), 175
work page 1998
-
[48]
C Pelachaud. 2009. Studies on gesture expressivity for a virtual agent. Speech Communication (2009)
work page 2009
-
[49]
Ildiko Pelczer, Francisco Cabiedes, and Fernando Gamboa. 2007. Expressions of Emotions in Virtual Agents: Empirical Evaluation. In Virtual Environments, Human-Computer Interfaces and Measurement Systems, 2007. VECIMS 2007. IEEE Symposium on. IEEE, 31–35
work page 2007
-
[50]
Christopher Peters, Catherine Pelachaud, Elisabetta Bevacqua, Maurizio Mancini, and Isabella Poggi. 2005. A model of attention and interest using gaze behavior. In International Workshop on Intelligent Virtual Agents . Springer, 229–240
work page 2005
-
[51]
Tanmay Randhavane, Aniket Bera, Kyra Kapsaskis, Kurt Gray, and Dinesh Manocha. 2019. FVA:Modeling Perceived Friendliness of Virtual Agents Us- ing Movement Characteristics. 2019 International Symposium on Mixed and Augmented Reality Special Issue of TVCG (2019)
work page 2019
-
[52]
Tanmay Randhavane, Aniket Bera, Emily Kubin, Kurt Gray, and Dinesh Manocha
-
[53]
Modeling Data-Driven Dominance Traits for Virtual Characters using Gait Analysis
Modeling Data-Driven Dominance Traits for Virtual Characters using Gait Analysis. arXiv preprint arXiv:1901.02037 (2019)
work page internal anchor Pith review Pith/arXiv arXiv 1901
-
[54]
Tanmay Randhavane, Aniket Bera, Emily Kubin, Austin Wang, Kurt Gray, and Dinesh Manocha. 2019. Pedestrian Dominance Modeling for Socially-Aware Robot Navigation. 2019 IEEE International Conference on Robotics and Automation (ICRA) (2019)
work page 2019
-
[55]
Tanmay Randhavane, Aniket Bera, and Dinesh Manocha. 2017. F2FCrowds: Plan- ning agent movements to enable face-to-face interactions. Presence: Teleoperators and Virtual Environments 26, 2 (2017), 228–246
work page 2017
-
[56]
Laurel D Riek, Tal-Chen Rabinowitch, Bhismadev Chakrabarti, and Peter Robin- son. 2009. How anthropomorphism affects empathy toward robots. InProceedings of the 4th ACM/IEEE international conference on Human robot interaction . ACM, 245–246
work page 2009
-
[57]
H. Riggio. 2017. Emotional Expressiveness. Encyclopedia of Personality and Individual Differences (2017)
work page 2017
-
[58]
Jesús J Rivas, Felipe Orihuela-Espina, L Enrique Sucar, Lorena Palafox, Jorge Hernández-Franco, and Nadia Bianchi-Berthouze. 2015. Detecting affective states in virtual rehabilitation. In Proceedings of the 9th International Conference on Pervasive Computing Technologies for Healthcare . ICST (Institute for Computer Sciences, Social-Informatics and âĂę, 287–292
work page 2015
-
[59]
J Russell. 1987. The identification of emotions from gait information. Journal of Nonverbal Beh. (1987)
work page 1987
-
[60]
Samuel S Sohn, Xun Zhang, Fernando Geraci, and Mubbasir Kapadia. 2018. An Emotionally Aware Embodied Conversational Agent. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems . Interna- tional Foundation for Autonomous Agents and Multiagent Systems, 2250–2252
work page 2018
-
[61]
Marcus Thiebaux, Stacy Marsella, Andrew N Marshall, and Marcelo Kallmann
-
[62]
Smartbody: Behavior realization for embodied conversational agents. In Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 151–158
-
[63]
Angela Tinwell, Mark Grimshaw, Debbie Abdel Nabi, and Andrew Williams. 2011. Facial expression of emotion and perception of the Uncanny Valley in virtual characters. Computers in Human Behavior 27, 2 (2011), 741–749
work page 2011
-
[64]
Jur Van den Berg, Ming Lin, and Dinesh Manocha. 2008. Reciprocal velocity ob- stacles for real-time multi-agent navigation. In2008 IEEE International Conference on Robotics and Automation . IEEE, 1928–1935
work page 2008
-
[65]
Gentiane Venture, Hideki Kadone, Tianxiang Zhang, Julie Grèzes, Alain Berthoz, and Halim Hicheur. 2014. Recognizing emotions conveyed by human gait. Inter- national Journal of Social Robotics 6, 4 (2014), 621–632
work page 2014
-
[66]
Maia Garau Anthony Steed Vinayagamoorthy, Vinoba and Mel Slater. 2004. An eye gaze model for dyadic interaction in an immersive virtual environment: Practice and experience. In Computer Graphics Forum
work page 2004
-
[67]
V. Vinayagamoorthy, M. Gillies, A. Steed, E. Tanguy, X. Pan, C. Loscos, and M. Slater. 2006. Building Expression into Virtual Characters. In Eurographics 2006 - State of the Art Reports , Brian Wyvill and Alexander Wilkie (Eds.). The Eurographics Association. https://doi.org/10.2312/egst.20061052 EVA: Generating Emotional Behavior of Virtual Agents using ...
-
[68]
Shihong Xia, Congyi Wang, Jinxiang Chai, and Jessica Hodgins. 2015. Realtime Style Transfer for Unlabeled Heterogeneous Human Motion. ACM Trans. Graph. 34, 4, Article 119 (July 2015), 10 pages. https://doi.org/10.1145/2766999
-
[69]
Heath Yates, Brent Chamberlain, Greg Norman, and William H. Hsu. 2017. Arousal Detection for Biometric Data in Built Environments using Machine Learn- ing. In Proceedings of IJCAI 2017 Workshop on Artificial Intelligence in Affective Computing (Proceedings of Machine Learning Research) , Neil Lawrence and Mark Reid (Eds.), Vol. 66. PMLR, 58–72. http://pro...
work page 2017
-
[70]
Katja Zibrek, Ludovic Hoyet, Kerstin Ruhland, and Rachel Mcdonnell. 2015. Exploring the effect of motion type and emotions on the perception of gender in virtual humans. ACM Transactions on Applied Perception (TAP) 12, 3 (2015), 11
work page 2015
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.