pith. sign in

arxiv: 1907.00377 · v1 · pith:XQX74V7Inew · submitted 2019-06-30 · 💻 cs.HC · cs.AI· cs.GR

FVA: Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics

Tanmay Randhavane , Aniket Bera , Kyra Kapsaskis , Kurt Gray , Dinesh Manocha This is my paper

Pith reviewed 2026-05-25 12:41 UTC · model grok-4.3

classification 💻 cs.HC cs.AIcs.GR
keywords virtual agentsaugmented realityfriendliness modelmovement characteristicssocial presencegaitsgesturesgazing
0
0 comments X

The pith

A data-driven model lets virtual agents move with gaits, gestures and gazing that users rate as friendlier in AR.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a method to improve how friendly virtual agents seem in augmented reality by adjusting their movements according to a model built from user studies and psychological characteristics. It applies the model to control gaits, gestures, and gazing as agents interact with a user's avatar and other agents. When tested, this produces higher average ratings for friendliness and social presence than agents without the adjustments. A sympathetic reader would care because such modeling could make virtual characters feel more approachable in mixed-reality experiences. The system produces movements fast enough for live use in AR setups.

Core claim

We present a new approach for improving the friendliness and warmth of a virtual agent in an AR environment by generating appropriate movement characteristics. Our algorithm is based on a novel data-driven friendliness model that is computed using a user-study and psychological characteristics. We use our model to control the movements corresponding to the gaits, gestures, and gazing of friendly virtual agents as they interact with the user's avatar and other agents in the environment. Our algorithm can generate plausible movements at interactive rates to increase the social presence. We also investigate the perception of a user in an AR setting and observe that an FVA has a statistically a

What carries the argument

The data-driven friendliness model computed from user-study data and psychological characteristics, which controls gaits, gestures, and gazing to produce movements rated as warmer.

Load-bearing premise

The friendliness model computed using a user-study and psychological characteristics accurately captures and predicts perceived friendliness for virtual agents interacting with the user's avatar and other agents in AR.

What would settle it

A replication user study in an AR setting that measures friendliness and social presence ratings for agents with and without the modeled movements and finds no statistically significant difference.

Figures

Figures reproduced from arXiv: 1907.00377 by Aniket Bera, Dinesh Manocha, Kurt Gray, Kyra Kapsaskis, Tanmay Randhavane.

Figure 1
Figure 1. Figure 1: Friendly Virtual Agent (FVA): We present an algorithm to model perceived friendliness of a virtual agent by varying its gaze (A), gait (B), and gestures corresponding to head nodding (C) and waving (D). These movement cues are generated using our novel data-driven friendliness model. We evaluate the benefits in terms of an improved sense of social presence using an AR validation study using a HoloLens. ABS… view at source ↗
Figure 2
Figure 2. Figure 2: Friendliness Model Overview: Friendliness can be conveyed using nonverbal cues of gaits, gestures, and gazing. We use data-driven techniques and psychological characterization to formulate a mapping between friendliness and gaits, gestures, and gazing [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Gait Visualizations: We present sample visualizations of three gaits from a publicly-available, motion-captured gait dataset used for our user-study. We asked the participants to rate each gait video on a 7-point scale for a friendliness measure (Section 4.1.3). Based on the responses to 49 gaits from participants, we designed a data-driven model of friendliness and gaits. where j is the gait ID, item is o… view at source ↗
Figure 4
Figure 4. Figure 4: Waving Gestures: We generated videos of virtual agents with nonverbal characteristics corresponding to varying levels of friend￾liness as predicted by our Friendliness Model. A closed gesture corresponds to a lower friendliness level, whereas an open gesture corresponds to a higher friendliness level. We performed a web-based validation study to evaluate our model using these videos. 4.5.1 Virtual Agents W… view at source ↗
Figure 6
Figure 6. Figure 6: Gazing Cues: Friendliness can be conveyed using non￾verbal cues associated with gazing. We model gazing features by computing the neck flexion and rotation angles such that the FVA maintains eye-contact with the user (represented by the rendering camera, ~pc). We can also check the line-of-sight between the agent and the user using visibility queries. For this computation, we assume that the FVA is facing … view at source ↗
read the original abstract

We present a new approach for improving the friendliness and warmth of a virtual agent in an AR environment by generating appropriate movement characteristics. Our algorithm is based on a novel data-driven friendliness model that is computed using a user-study and psychological characteristics. We use our model to control the movements corresponding to the gaits, gestures, and gazing of friendly virtual agents (FVAs) as they interact with the user's avatar and other agents in the environment. We have integrated FVA agents with an AR environment using with a Microsoft HoloLens. Our algorithm can generate plausible movements at interactive rates to increase the social presence. We also investigate the perception of a user in an AR setting and observe that an FVA has a statistically significant improvement in terms of the perceived friendliness and social presence of a user compared to an agent without the friendliness modeling. We observe an increment of 5.71% in the mean responses to a friendliness measure and an improvement of 4.03% in the mean responses to a social presence measure.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper presents FVA, a data-driven friendliness model for virtual agents in AR that modulates gait, gesture, and gaze parameters based on a user study combined with psychological characteristics. The model is used to generate movements for agents interacting with the user's avatar and other agents on a Microsoft HoloLens, with the central claim being that this produces statistically significant gains in perceived friendliness (5.71% mean increase) and social presence (4.03% mean increase) relative to agents without the modeling.

Significance. If the model construction and validation hold, the work offers a practical, real-time method for increasing social warmth in AR agents via movement characteristics, which could inform design guidelines in HCI for more engaging virtual interactions. The HoloLens integration demonstrates feasibility at interactive rates, a strength for applied AR research.

major comments (2)
  1. [Abstract] Abstract: The friendliness model is stated to be 'computed using a user-study and psychological characteristics,' yet the manuscript provides no information on the fitting procedure, feature selection, regularization, or any validation metrics such as cross-validation, out-of-sample correlation, or R² values. This is load-bearing for the central claim, as the reported perceptual gains could stem from study confounds rather than the modeled movements.
  2. [Abstract] Abstract: The user-study results assert 'statistically significant improvement' with specific mean increments (5.71% friendliness, 4.03% social presence), but omit participant count, statistical test details, p-values, confidence intervals, or error bars. Without these, the effect sizes and reliability cannot be assessed.
minor comments (1)
  1. [Abstract] The abstract refers to 'plausible movements at interactive rates' but does not report frame rates, latency measurements, or hardware benchmarks for the HoloLens implementation.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments on the abstract. We agree that additional details on model construction and statistical reporting will strengthen the paper and will revise the abstract and main text accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The friendliness model is stated to be 'computed using a user-study and psychological characteristics,' yet the manuscript provides no information on the fitting procedure, feature selection, regularization, or any validation metrics such as cross-validation, out-of-sample correlation, or R² values. This is load-bearing for the central claim, as the reported perceptual gains could stem from study confounds rather than the modeled movements.

    Authors: We agree the abstract (and potentially the main text) should provide more explicit information on these aspects. In the revision we will add a concise description of the fitting procedure, feature selection, regularization, and validation metrics (including cross-validation and R²) to the abstract and ensure the Methods section contains the full details. revision: yes

  2. Referee: [Abstract] Abstract: The user-study results assert 'statistically significant improvement' with specific mean increments (5.71% friendliness, 4.03% social presence), but omit participant count, statistical test details, p-values, confidence intervals, or error bars. Without these, the effect sizes and reliability cannot be assessed.

    Authors: We agree the abstract should report these key statistics for transparency. In the revision we will add the participant count, the statistical test used, p-values, confidence intervals, and a reference to the error bars shown in the results figures. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected in derivation chain

full rationale

The paper constructs its friendliness model from an independent user study plus psychological characteristics, then applies the model to generate agent movements and reports perceptual gains from a separate AR evaluation study measuring friendliness and social presence via mean response improvements. No equations, parameters, or steps in the abstract reduce any claimed prediction or result to the inputs by construction (e.g., no self-definitional mappings, fitted inputs renamed as predictions, or load-bearing self-citations). The chain is empirically grounded in distinct data collections rather than tautological or self-referential reductions.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit details on free parameters, axioms, or invented entities; the model is presented as computed from user-study data and psychological characteristics.

pith-pipeline@v0.9.0 · 5724 in / 1017 out tokens · 43144 ms · 2026-05-25T12:41:33.846058+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 59 canonical work pages · 1 internal anchor

  1. [1]

    http://mocap.cs.cmu

    Cmu graphics lab motion capture database. http://mocap.cs.cmu. edu/, 2018

    work page 2018

  2. [2]

    A. E. Abele. The dynamics of masculine-agentic and feminine- communal traits: findings from a prospective study. Journal of person- ality and social psychology, 85(4):768, 2003

    work page 2003

  3. [3]

    Amores, X

    J. Amores, X. Benavides, M. Comin, A. Fuste, P. Pla, and D. Miralles. Smart avatars: using avatars to interact with objects. In Proceedings of ACM CHI, pp. 1–3, 2012

    work page 2012

  4. [4]

    M. Argyle. Bodily Communication. 1998

    work page 1998

  5. [5]

    Argyle, L

    M. Argyle, L. Lefebvre, and M. Cook. The meaning of five patterns of gaze. European journal of social psychology , 4(2):125–136, 1974

    work page 1974

  6. [6]

    J. N. Bailenson, A. C. Beall, J. Blascovich, M. Raimundo, and M. Weis- buch. Intelligent agents who wear your face: Users reactions to the virtual self. In International Workshop on Intelligent Virtual Agents, pp. 86–99. Springer, 2001

    work page 2001

  7. [7]

    N. Bee, C. Pollock, E. Andr ´e, and M. Walker. Bossy or wimpy: ex- pressing social dominance by combining gaze and linguistic behaviors. In International Conference on Intelligent Virtual Agents , pp. 265–271. Springer, 2010

    work page 2010

  8. [8]

    Bente, S

    G. Bente, S. R¨uggenberg, N. C. Kr¨amer, and F. Eschenburg. Avatar- mediated networking: Increasing social presence and interpersonal trust in net-based collaborations. Human communication research , 34(2):287–318, 2008

    work page 2008

  9. [9]

    A. Bera, T. Randhavane, E. Kubin, H. Shaik, K. Gray, and D. Manocha. Data-driven modeling of group entitativity in virtual environments. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, p. 31. ACM, 2018

    work page 2018

  10. [10]

    A. Bera, T. Randhavane, R. Prinja, K. Kapsaskis, A. Wang, K. Gray, and D. Manocha. The emotionally intelligent robot: Improving social navigation in crowded environments. arXiv preprint arXiv:1903.03217, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1903

  11. [11]

    Bergmann, F

    K. Bergmann, F. Eyssel, and S. Kopp. A second chance to make a first impression? how appearance and nonverbal behavior affect perceived warmth and competence of virtual agents over time. In International Conference on Intelligent Virtual Agents, pp. 126–138. Springer, 2012

    work page 2012

  12. [12]

    R.-J. Beun, E. De V os, and C. Witteman. Embodied conversational agents: effects on memory performance and anthropomorphisation. In International Workshop on Intelligent Virtual Agents, pp. 315–319. Springer, 2003

    work page 2003

  13. [13]

    Cafaro, H

    A. Cafaro, H. H. Vilhj ´almsson, T. Bickmore, D. Heylen, K. R. J´ohannsd´ottir, and G. S. Valgar¨osson. First impressions: Users judg- ments of virtual agents personality and interpersonal attitude in first encounters. In International conference on intelligent virtual agents , pp. 67–80. Springer, 2012

    work page 2012

  14. [14]

    L. L. Carli, S. J. LaFleur, and C. C. Loeber. Nonverbal behavior, gender, and influence. Journal of personality and social psychology , 68(6):1030, 1995

    work page 1995

  15. [15]

    Chuah, A

    J. Chuah, A. Robb, C. White, A. Wendling, S. Lampotang, R. Kopper, and B. Lok. Exploring agent physicality and social presence for medical team training. Presence: Teleoperators and Virtual Environments , 22:141–170, 08 2013. doi: 10.1162/PRES a 00145

    work page doi:10.1162/pres 2013

  16. [16]

    A. J. Cuddy, S. T. Fiske, and P. Glick. Warmth and competence as universal dimensions of social perception: The stereotype content model and the bias map. Advances in Experimental Social Psychology , 40:61–149, 2008

    work page 2008

  17. [17]

    A. J. Cuddy, P. Glick, and A. Beninger. The dynamics of warmth and competence judgments, and their outcomes in organizations. Research in organizational behavior, 31:73–98, 2011

    work page 2011

  18. [18]

    Curtis, A

    S. Curtis, A. Best, and D. Manocha. Menge: A modular framework for simulating crowd movement. Collective Dynamics, 1:1–40, 2016

    work page 2016

  19. [19]

    Duchan, M

    J. Duchan, M. Maxwell, and D. Kovarsky. Evaluating competence in the course of everyday interaction. In D. Kovarsky, J. Duchan, and M. Maxwell, eds., Constructing (in)competence, chap. 1, pp. 3–26. Lawrence Erlbaum, Mahwah, NJ, 1999

    work page 1999

  20. [20]

    Ferstl and R

    Y . Ferstl and R. McDonnell. Investigating the use of recurrent motion modelling for speech gesture generation. In Proceedings of the 18th In- ternational Conference on Intelligent Virtual Agents, IVA 2018, Sydney, NSW, Australia, November 05-08, 2018, pp. 93–98, 2018

    work page 2018

  21. [21]

    S. T. Fiske, A. J. Cuddy, and P. Glick. Universal dimensions of social cognition: Warmth and competence. Trends in cognitive sciences , 11(2):77–83, 2007

    work page 2007

  22. [22]

    S. T. Fiske, A. J. C. Cuddy, P. Glick, and J. Xu. Journal of Personality and Social Psychology, 82(6):878–902, 2002

    work page 2002

  23. [23]

    Forlizzi, J

    J. Forlizzi, J. Zimmerman, V . Mancuso, and S. Kwak. How interface agents affect interaction between humans and computers. In Proceed- ings of the 2007 conference on Designing pleasurable products and interfaces, pp. 209–221. ACM, 2007

    work page 2007

  24. [24]

    Fraser, I

    J. Fraser, I. Papaioannou, and O. Lemon. Spoken conversational ai in video games–emotional dialogue management increases user en- gagement. In Proceedings of the 18th International Conference on Intelligent Virtual Agents, pp. 179–184. ACM, 2018

    work page 2018

  25. [25]

    Gerhard, D

    M. Gerhard, D. J. Moore, and D. J. Hobbs. Continuous presence in collaborative virtual environments: Towards a hybrid avatar-agent model for user representation. In International Workshop on Intelligent Virtual Agents, pp. 137–155. Springer, 2001

    work page 2001

  26. [26]

    R. G. Harper. Power, dominance, and nonverbal behavior: An overview. In Power , dominance, and nonverbal behavior, pp. 29–48. Springer, 1985

    work page 1985

  27. [27]

    Hasegawa, N

    D. Hasegawa, N. Kaneko, S. Shirakawa, H. Sakuta, and K. Sumi. Evaluation of speech-to-gesture generation using bi-directional lstm network. In Proceedings of the 18th International Conference on Intelligent Virtual Agents, pp. 79–86. ACM, 2018

    work page 2018

  28. [28]

    Holden, T

    D. Holden, T. Komura, and J. Saito. Phase-functioned neural networks for character control. ACM Transactions on Graphics (TOG), 36(4):42, 2017

    work page 2017

  29. [29]

    Holden, J

    D. Holden, J. Saito, and T. Komura. A deep learning framework for character motion synthesis and editing. ACM Transactions on Graphics (TOG), 35(4):138, 2016

    work page 2016

  30. [30]

    Huang, T

    C.-M. Huang, T. Iio, S. Satake, and T. Kanda. Modeling and controlling friendliness for an interactive museum robot. In Robotics: Science and Systems, 2014

    work page 2014

  31. [31]

    K. Kim, L. Boelling, S. Haesler, J. Bailenson, G. Bruder, and G. F. Welch. Does a digital assistant need a body? the influence of visual embodiment and social behavior on the perception of intelligent virtual agents in ar. In 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 105–114. IEEE, 2018

    work page 2018

  32. [32]

    K. Kim, G. Bruder, and G. Welch. Exploring the effects of observed physicality conflicts on real-virtual human interaction in augmented reality. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology, p. 31. ACM, 2017

    work page 2017

  33. [33]

    Kim and G

    K. Kim and G. Welch. Maintaining and enhancing human-surrogate presence in augmented reality. In 2015 IEEE International Symposium on Mixed and Augmented Reality Workshops , pp. 15–19. IEEE, 2015

    work page 2015

  34. [34]

    T. Koda, T. Hirano, and T. Ishioh. Development and perception evalua- tion of culture-specific gaze behaviors of virtual agents. InInternational Conference on Intelligent Virtual Agents, pp. 213–222. Springer, 2017

    work page 2017

  35. [35]

    N. C. Kr¨amer, B. Karacora, G. Lucas, M. Dehghani, G. R ¨uther, and J. Gratch. Closing the gender gap in stem with friendly male instruc- tors? on the effects of rapport behavior and gender of a virtual agent in an instructional interaction. Computers & Education, 99:1–13, 2016

    work page 2016

  36. [36]

    Laban and F

    R. Laban and F. C. Lawrence. Effort: economy of human movement . Macdonald & Evans, 1979

    work page 1979

  37. [37]

    M. Lee, K. Kim, S. Daher, A. Raij, R. Schubert, J. Bailenson, and G. Welch. The wobbly table: Increased social presence via subtle incidental movement of a real-virtual table. In 2016 IEEE Virtual Reality (VR), pp. 11–17. IEEE, 2016

    work page 2016

  38. [38]

    C. Li, T. Androulakaki, A. Y . Gao, F. Yang, H. Saikia, C. Peters, and G. Skantze. Effects of posture and embodiment on social distance in human-agent interaction in mixed reality. In Proceedings of the 18th International Conference on Intelligent Virtual Agents , pp. 191–196. ACM, 2018

    work page 2018

  39. [39]

    H. Li. Photorealistic facial digitization and manipulation: technical perspective. Communications of the ACM, 62(1):95–95, 2018

    work page 2018

  40. [40]

    S. Lin, Y . Chen, Y .-K. Lai, R. R. Martin, and Z.-Q. Cheng. Fast capture of textured full-body avatar with rgb-d cameras. The Visual Computer, 32(6-8):681–691, 2016

    work page 2016

  41. [41]

    Lombard, T

    M. Lombard, T. B. Ditton, and L. Weinstein. Measuring presence: The temple presence inventory. In In International Workshop on Presence, p. 115, 2009

    work page 2009

  42. [42]

    Lombardi, J

    S. Lombardi, J. Saragih, T. Simon, and Y . Sheikh. Deep appearance models for face rendering. ACM Transactions on Graphics (TOG) , 37(4):68, 2018

    work page 2018

  43. [43]

    Montepare and L

    J. Montepare and L. Zebrowitz-McArthur. Impressions of people created by age-related qualities of their gaits. Journal of personality and social psychology, 55:547–56, 11 1988. doi: 10.1037/0022-3514. 55.4.547

    work page doi:10.1037/0022-3514 1988

  44. [44]

    M. E. Morris, B. Adair, K. Miller, E. Ozanne, R. Hansen, A. J. Pearce, N. Santamaria, L. Viega, M. Long, and C. M. Said. Smart-home technologies to assist older people to live well at home. Journal of aging science, 1(1):1–9, 2013

    work page 2013

  45. [45]

    Mousas, D

    C. Mousas, D. Anastasiou, and O. Spantidi. The effects of appearance and motion of virtual characters on emotional reactivity. Computers in Human Behavior, 86:99–108, 2018

    work page 2018

  46. [46]

    Nagano, J

    K. Nagano, J. Seo, J. Xing, L. Wei, Z. Li, S. Saito, A. Agarwal, J. Fur- sund, H. Li, R. Roberts, et al. pagan: real-time avatars using dynamic textures. In SIGGRAPH Asia 2018 Technical Papers , p. 258. ACM, 2018

    work page 2018

  47. [47]

    Narang, A

    S. Narang, A. Best, T. Randhavane, A. Shapiro, and D. Manocha. Pedvr: Simulating gaze-based interactions between a real user and virtual crowds. In Proceedings of the 22nd ACM conference on virtual reality software and technology, pp. 91–100. ACM, 2016

    work page 2016

  48. [48]

    T.-H. D. Nguyen, E. Carstensdottir, N. Ngo, M. S. El-Nasr, M. Gray, D. Isaacowitz, and D. Desteno. Modeling warmth and competence in virtual characters. In International Conference on Intelligent Virtual Agents, pp. 167–180. Springer, 2015

    work page 2015

  49. [49]

    Normoyle, F

    A. Normoyle, F. Liu, M. Kapadia, N. I. Badler, and S. J¨org. The effect of posture and dynamics on the perception of emotion. In Proceedings of the ACM Symposium on Applied Perception , pp. 91–98. ACM, 2013

    work page 2013

  50. [50]

    Norouzi, K

    N. Norouzi, K. Kim, J. Hochreiter, M. Lee, S. Daher, G. Bruder, and G. Welch. A systematic survey of 15 years of user studies published in the intelligent virtual agents conference. In Proceedings of the 18th international conference on intelligent virtual agents, pp. 17–22. ACM, 2018

    work page 2018

  51. [51]

    G. M. O’Hare, A. G. Campbell, J. W. Stafford, and R. Aiken. Nexus: Behavioural realism in mixed reality scenarios through virtual sensing. In 18th Annual Conference on Computer Animation and Social Agents (CASA 2005), Hong Kong, China, October 17-25, 2005 , 2005

    work page 2005

  52. [52]

    Randhavane, A

    T. Randhavane, A. Bera, E. Kubin, A. Wang, K. Gray, and D. Manocha. Pedestrian dominance modeling for socially-aware robot navigation. 2019 IEEE International Conference on Robotics and Automation (ICRA), 2019

    work page 2019

  53. [53]

    Randhavane, A

    T. Randhavane, A. Bera, and D. Manocha. F2fcrowds: Planning agent movements to enable face-to-face interactions.Presence: Teleoperators and Virtual Environments, 26(2):228–246, 2017

    work page 2017

  54. [54]

    M. M. Reece and R. N. Whitman. Expressive movements, warmth, and verbal reinforcement. The Journal of Abnormal and Social Psychology , 64(3):234, 1962

    work page 1962

  55. [55]

    D. M. Romano, G. Sheppard, J. Hall, A. Miller, and Z. Ma. Basic: A believable, adaptable socially intelligent character for social presence. In Proceedings of the 8th Annual International Workshop on Presence , pp. 287–290. Citeseer, 2005

    work page 2005

  56. [56]

    P. Rubin. Facebook can make vr avatars look and move exactly like you, 2019

    work page 2019

  57. [57]

    J. Tong, J. Zhou, L. Liu, Z. Pan, and H. Yan. Scanning 3d full hu- man bodies using kinects. IEEE Transactions on Visualization and Computer Graphics, 18(4):643–650, 2012

    work page 2012

  58. [58]

    Youngblut

    C. Youngblut. Experience of presence in virtual environments. Techni- cal report, Institute For Defense Analyses, Alexandria, V A, 2003

    work page 2003

  59. [59]

    Zibrek, E

    K. Zibrek, E. Kokkinara, and R. McDonnell. The effect of realistic appearance of virtual characters in immersive environments-does the character’s personality play a role? IEEE transactions on visualization and computer graphics, 24(4):1681–1690, 2018

    work page 2018