Pith. sign in

REVIEW 5 major objections 5 minor 61 references

Understanding Emotional Body Expressions via Large Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a large language model can recognize emotions from 3D full-body skeleton data and generate textual explanations by converting the skeleton into specialized tokens.

desk verdict Solid engineering first pass at LLM-based skeleton emotion recognition and explanation; the recognition claim holds, but the description results are weaker than advertised because the gold emotion label is part of the prompt. read the letter →

arxiv 2412.12581 v2 pith:66GOTM34 submitted 2024-12-17 cs.HC

Haifeng Lu , Jiuyi Chen , Feng Liang , Mingkui Tan , Runhao Zeng , Xiping Hu This is my paper · ORCID
classification cs.HC
keywords emotionrecognition3Dskeletonlargelanguagemodelsbodymovementmulti-granularitytokenizationcontrastivelearninghuman-computerinteractionexplainable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EAI-LLM treats a 3D skeleton sequence as a kind of language: a multi-granularity tokenizer turns body movement into semantic, spatial, and temporal tokens, and a masked unified token module lets datasets with different joint counts and frame lengths be trained together. The same model then both outputs an emotion label and writes a short explanation of why, using the language model's pretrained knowledge to compensate for very little labeled skeleton-description data. If the central claim holds, emotion recognition from body movement becomes explainable rather than a black-box label, which matters for human-computer interaction at a distance or without microphones.

What carries the argument

The central machinery is the Multi-Granularity Skeleton Tokenizer (MGST), which extracts a semantic token summarizing the whole body motion plus spatial and temporal tokens that preserve joint and frame detail, giving the LLM diverse evidence for generation. A Unified Skeleton Token (UST) module applies an attention mask to pad all tokens to one length, so datasets of different skeleton topologies are pooled like sentences of different lengths; a skeleton-text contrastive loss, using a CLIP text encoder and KL divergence, aligns skeleton features with language space, and LoRA fine-tuning adapts the frozen LLaMA-7B base while keeping its pretrained weights intact.

What would settle it

On the held-out test split, run the same prompts after replacing the skeleton-token block with a constant vector of matched length, or after randomly permuting joint coordinates within each frame; if recognition accuracy stays near the reported 85.44%, 71.17%, and 66.97% values while generated descriptions remain fluent, the skeleton input is not what carries the claimed understanding.

Watch

Extended reading notes

Core claim

The paper's central claim is that 3D skeleton sequences can be tokenized into LLM-compatible inputs so that one fine-tuned language model performs emotion classification and generates fine-grained emotion descriptions at the same time. Pre-training aligns skeleton features with text features through contrastive learning, and masking normalizes heterogeneous tokens so that the Emilya, KDAE, and EGBM datasets can be jointly trained; on the authors' re-implemented comparisons the model reaches 85.44% accuracy on Emilya, 71.17% on KDAE, and 66.97% on EGBM, close to dedicated graph-convolutional recognizers, while its descriptions score higher on Rouge, BLEU, and METEOR than GPT-4 and Gemini prompted with the same skeletons rendered as video.

Load-bearing premise

The whole method rests on the assumption that the model's emotion judgments are actually driven by the skeleton tokens rather than by the language model's pretrained expectations about emotion labels, because no control with text-only prompts or scrambled skeleton features is reported.

Editorial extensions

If this is right

  • If correct, the same model can label an emotion and explain its bodily cues from pure 3D joint coordinates, no face or voice required, which suits cameras placed far from the user.
  • Joint training across heterogeneous skeleton datasets becomes feasible via length masking, and the authors report an average accuracy gain of about 4.5% for semantic tokens over separate training.
  • The language model's prior knowledge supports fluent emotion descriptions from only about 279 manually annotated skeleton-description pairs, suggesting data-efficient explainability.
  • Spatio-temporal tokens are better for description quality while semantic tokens are better for recognition, so the token granularity can be chosen by task.
  • Training order matters: fine-tuning recognition first and description second causes a large drop in recognition accuracy, which the authors attribute to conflicting output-format demands.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported accuracy could partly come from the language model's pretrained prior over emotion labels rather than from the skeleton input; a control with no skeleton tokens or with shuffled joint coordinates would measure how much the motion actually contributes.
  • A natural extension is an automatic consistency check that verifies each generated body cue (head down, hands covering face) against the actual joint geometry, which could also mitigate the hallucination examples the paper itself reports.
  • The same tokenize-align-and-finetune recipe could transfer to other continuous non-text signals such as gait, gesture, or physiological streams, because the paper's alignment step is modality-agnostic in principle.
  • Because descriptions were trained on a small hand-labeled subset, scaling the annotation with a richer emotion-action vocabulary is a testable path to more detailed explanations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript proposes EAI-LLM, a framework that feeds 3D skeleton sequences into LLaMA via a GCN-based encoder and learned linear projections, with LoRA fine-tuning, to perform emotion classification and generate textual explanations. It introduces a multi-granularity skeleton tokenizer, a unified skeleton token module for joint training across heterogeneous datasets, and a skeleton-text contrastive loss with a KL-divergence formulation. Experiments on Emilya, KDAE, and EGBM report recognition accuracies of 85.44%, 71.17%, and 66.97% respectively, which are comparable to or slightly below state-of-the-art baselines. For emotion description, the reported Rouge/BLEU/METEOR scores are higher than those of GPT-4 and Gemini 1.5 Pro prompted with videos rendered from the same skeletons. The paper includes ablations on architecture, skeleton encoder, LLM choice, output format, and training order, and closes with a limitation section that acknowledges hallucination issues.

Significance. If the central causal claim is established, this is a useful contribution to interpretable skeleton-based emotion recognition: it demonstrates a concrete way to bridge 3D skeleton data and LLMs, enabling both classification and textual explanation. The paper's strengths include a broad experiment suite across three datasets, a detailed ablation appendix, an honest limitations section, and a direct comparison with mainstream multimodal LLMs. However, the central attribution of recognition performance to the skeleton tokens is not yet demonstrated: no text-only or random-token control is reported, and the description evaluation is conditioned on the gold emotion label rather than the model's own prediction. These gaps are load-bearing for the paper's main claims, but they are fixable with additional experiments and clarifications, so the current value is conditional on those additions.

major comments (5)
  1. [§4.3, Table 4] The central claim that emotion recognition is driven by the skeleton tokens is not isolated. There is no control condition with the same LoRA-tuned LLaMA and prompt but with the skeleton tokens removed, shuffled, or replaced by random features. Appendix B shows that raw skeleton encoder features yield 46.29% accuracy versus 85.44% with CE+contrastive alignment, but this compares different feature qualities, not the contribution of the skeleton input relative to the LLM's prior. A text-only baseline (e.g., the same prompt with no <SkeletonFeature> or with fixed random vectors) is needed to measure the marginal gain from skeleton tokens; without it, the reported accuracies in Table 4 could partly reflect the model's prior over emotion labels and prompt template. This is load-bearing for the abstract's claim that the model understands emotional body expressions.
  2. [§3.3, Table 5] The emotion-description prompt supplies the gold label: "The emotion of this person is [shame], please tell me some reasons for it." Consequently, Table 5 evaluates explanation of a known label, not generation from the skeleton alone. An end-to-end evaluation that uses the label predicted by the recognition stage is missing. Moreover, the comparison with GPT-4 and Gemini in Table 5 needs to state explicitly whether those models also received the gold label in the prompt; the example in Figure 4 suggests they did, but the table caption and text are ambiguous. This is load-bearing for the claim that EAI-LLM generates descriptions based on its own classification results.
  3. [§4.4, Table 3] The table is difficult to interpret and the results undermine the "simultaneously" claim. The R→D rows show a catastrophic drop in recognition accuracy (the text reports relative drops of 37.06% and 61.86%), meaning that after sequential fine-tuning the model cannot perform both tasks without degradation. The paper should either present a multi-task training scheme that avoids this effect, or clearly state that the two capabilities require separately fine-tuned models. Additionally, the table layout appears misaligned (e.g., the row "Semantic R→D" contains entries that do not match the column structure), making it impossible to verify the reported averages and drops.
  4. [§4.2, Evaluation Protocols] Recognition results are based on a single random 4:1 split with no error bars, repeated runs, or significance tests. For a claim of "comparable to existing methods," the lack of variance reporting is a serious gap, especially because the improvements over baselines on KDAE and EGBM are small (e.g., 71.17% vs. 70.46% for CTR-GCN on KDAE). Please report mean ± std over at least three random seeds, or a paired significance test, to support the comparative claims.
  5. [§4.1, Table 3] The emotion-description subset consists of 174 Emilya samples and 105 KDAE samples, with no EGBM descriptions mentioned. Yet Table 3 reports description metrics "averaged across the three datasets." Please clarify how EGBM contributes to the description evaluation, or restrict the reported averages to the two datasets that actually have description annotations.
minor comments (5)
  1. [§3.2, Eq. (1)] Equation (1) is described as applying "a mask to all skeleton tokens," but the actual operation is padding masking to retain original tokens and nullify padded elements. Please rephrase to clarify that the mask applies only to padded positions.
  2. [§4.5] The text states that all baselines were re-implemented and preprocessing was standardized, but no details are given about the re-implementation (hyperparameters, training epochs, validation splits). Please provide these details in the supplementary material to support the claim of fair comparison.
  3. [Figure 3] The confusion matrices contain unrendered Uni* encoding sequences (e.g., "/uni00000024/..."), indicating a PDF encoding problem. Please replace these with proper class labels.
  4. [§4.4] The bullet claiming that "semantic tokens offer a balanced trade-off between recognition accuracy and description capability" is hard to reconcile with Table 3, where spatio-temporal tokens show much higher Rouge and BLEU scores in the D→R condition. Please restate or qualify this conclusion.
  5. [§1, Contributions] The contribution list calls EAI-LLM "the first approach" to use LLMs for skeleton-based emotion recognition with descriptions. This is a strong novelty claim that is not supported by a systematic comparison with prior work; consider softening it to "to our knowledge" and citing the most related efforts.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: the pipeline is standard supervised learning with held-out evaluation; self-citations are background only.

full rationale

The paper's derivation chain is self-contained rather than circular. Skeleton tokens come from an encoder pre-trained with cross-entropy and contrastive losses against external text embeddings (Section 3.4), and the LLaMA decoder is LoRA fine-tuned and evaluated on held-out splits (Section 4.2, Tables 4, 6, and Appendix B). No equation defines a predicted quantity in terms of the target: recognition accuracy is measured against external ground-truth labels, and description metrics are compared with human-written references. The self-citations (Lu, Hu, and Hu 2023; Zeng et al. 2024; You et al. 2024) support background claims about 3D skeleton robustness and neural-net performance; none carries the central argument. The emotion-description prompt includes the gold label ('The emotion of this person is shame...'), so the description results in Table 5 should be read as label-conditioned explanation rather than end-to-end description from skeleton alone; however, that is an evaluation-scope limitation, not a circular reduction, since the explanation target is an external human text and not the model's own predicted output. Appendix B's raw-encoder baseline (46.29%) versus the aligned-token result (85% on Emilya) shows the skeleton-token representation carries signal beyond a text prior. No step reduces to its input by construction, and no result is forced by a self-citation chain.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The paper introduces architectural modules (MGST, UST) rather than new physical entities. The empirical claims rest on standard deep-learning assumptions about encoder quality, LLM priors, and annotation correctness. The main load-bearing choices are the frame-length normalization, LoRA settings, and contrastive temperature, none of which are derived from first principles.

free parameters (5)
  • Maximum token length L (frame normalization to 64) = 64 frames
    All skeleton sequences are padded or downsampled to 64 frames before tokenization (Section 4.2); the choice affects token length and the UST mask design.
  • LoRA rank and alpha = r=64, alpha=16
    LoRA hyperparameters for LLaMA-7B fine-tuning (Section 4.2); Appendix C shows LLaMA-13B required r=128, alpha=32 to improve, indicating sensitivity.
  • Contrastive temperature tau = not reported
    Temperature in the skeleton-text contrastive loss, Eq. (3); chosen by hand but the value is not stated in the paper.
  • Number of linear projection layers = 1
    Selected via ablation (Appendix A); more layers reduce accuracy from 80.63% to 78.26%.
  • Spatial/temporal pooling configurations = TP, SP, STP with 1x1 convolutions
    Pooling choices in the MGST tokenizer (Section 3.1) determine token dimensions and the information retained for generation versus classification.
assumptions (3)
  • domain assumption The CTR-GCN skeleton encoder features capture emotion-relevant body movement information.
    Invoked in Sections 3.1 and 3.4 as the source of skeleton features; if the encoder discards expressive cues, the tokens cannot support recognition or description.
  • domain assumption Pre-trained LLaMA and CLIP models contain useful background knowledge about emotional body language.
    The method relies on LLM prior knowledge (Sections 1 and 3.3) and CLIP text embeddings (Section 3.4); no control isolates how much priors contribute to the final accuracy.
  • domain assumption The manually annotated emotion descriptions are accurate ground truth for the description task.
    Section 4.1 describes 279 annotated samples; if the annotations are biased or noisy, the Rouge/BLEU/METEOR scores are not meaningful.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Understanding Emotional Body Expressions via Large Language Models." pith.science (2026). https://pith.science/paper/66GOTM34

@misc{pith2026241212581,
  author       = {Pith},
  title        = {Pith review of: Understanding Emotional Body Expressions via Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/66GOTM34}},
  note         = {Machine review of arXiv:2412.12581}
}
read the original abstract

Emotion recognition based on body movements is vital in human-computer interaction. However, existing emotion recognition methods predominantly focus on enhancing classification accuracy, often neglecting the provision of textual explanations to justify their classifications. In this paper, we propose an Emotion-Action Interpreter powered by Large Language Model (EAI-LLM), which not only recognizes emotions but also generates textual explanations by treating 3D body movement data as unique input tokens within large language models (LLMs). Specifically, we propose a multi-granularity skeleton tokenizer designed for LLMs, which separately extracts spatio-temporal tokens and semantic tokens from the skeleton data. This approach allows LLMs to generate more nuanced classification descriptions while maintaining robust classification performance. Furthermore, we treat the skeleton sequence as a specific language and propose a unified skeleton token module. This module leverages the extensive background knowledge and language processing capabilities of LLMs to address the challenges of joint training on heterogeneous datasets, thereby significantly enhancing recognition accuracy on individual datasets. Experimental results demonstrate that our model achieves recognition accuracy comparable to existing methods. More importantly, with the support of background knowledge from LLMs, our model can generate detailed emotion descriptions based on classification results, even when trained on a limited amount of labeled skeleton data.

Figures

Figures reproduced from arXiv: 2412.12581 by the authors.

Figure 1
Figure 1. This work presents a novel approach for 3D full-body skeleton-based emotion recognition using fine-tuned LLMs, [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Diagram of skeleton-language alignment. Skeleton Encoder Our skeleton encoder is designed in a manner similar to CTR-GCN (Chen et al. 2021), enabling the extraction of skeleton features F s . Following pooling and dimensional transformation, we derive a C-dimensional feature vector z s . The backbone of the encoder is flexible and can be substituted with other GCN-based networks, such as HD-GCN (Lee et al. 2023) or … view at source ↗
Figure 3
Figure 3. Confusion matrices for Emilya dataset using dif [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Examples for emotion description capabilities of [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 45 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Alayrac, J.-B.; Donahue, J.; Luc, P.; Miech, A.; Barr, I.; Hasson, Y.; Lenc, K.; Mensch, A.; Millican, K.; Reynolds, M.; et al. 2022. Flamingo: a visual language model for few-shot learning. NeurIPS, 35: 23716--23736

    work page 2022

  4. [4]

    Anthropic. 2024. Claude: An AI Assistant by Anthropic. https://www.anthropic.com/product. Accessed: 2024-08-01

    work page 2024

  5. [5]

    Beyan, C.; Karumuri, S.; Volpe, G.; Camurri, A.; and Niewiadomski, R. 2023. Modeling Multiple Temporal Scales of Full-Body Movements for Emotion Classification. IEEE Transactions on Affective Computing, 14(2): 1070--1081

    work page 2023

  6. [6]

    D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al

    Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. NeurIPS, 33: 1877--1901

    2020

  7. [7]

    Chen, Y.; Zhang, Z.; Yuan, C.; Li, B.; Deng, Y.; and Hu, W. 2021. Channel-wise topology refinement graph convolution for skeleton-based action recognition. In ICCV, 13359--13368

    work page 2021

  8. [8]

    E.; Stoica, I.; and Xing, E

    Chiang, W.-L.; Li, Z.; Lin, Z.; Sheng, Y.; Wu, Z.; Zhang, H.; Zheng, L.; Zhuang, S.; Zhuang, Y.; Gonzalez, J. E.; Stoica, I.; and Xing, E. P. 2023. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90\ Quality

    work page 2023

Show all 61 references
  1. [9]

    W.; Sutton, C.; Gehrmann, S.; et al

    Chowdhery, A.; Narang, S.; Devlin, J.; Bosma, M.; Mishra, G.; Roberts, A.; Barham, P.; Chung, H. W.; Sutton, C.; Gehrmann, S.; et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240): 1--113

    2023

  2. [10]

    Daoudi, M.; Berretti, S.; Pala, P.; Delevoye, Y.; and Del Bimbo, A. 2017. Emotion recognition by body movement representation on the manifold of symmetric positive definite matrices. In ICIAP, 550--560

    2017

  3. [11]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805

    2018 arXiv

  4. [12]

    S.; and Karray, F

    El Ayadi, M.; Kamel, M. S.; and Karray, F. 2011. Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern recognition, 44(3): 572--587

    2011

  5. [13]

    Fourati, N.; and Pelachaud, C. 2015. Multi-level classification of emotional body expression. In 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), volume 1, 1--8

    2015

  6. [14]

    Fourati, N.; and Pelachaud, C. 2016. Perception of emotions and body movement in the emilya database. IEEE Transactions on Affective Computing, 9(1): 90--101

    2016

  7. [15]

    Fourati, N.; Pelachaud, C.; and Darmon, P. 2019. Contribution of temporal and multi-level body cues to emotion classification. In 2019 8th International Conference on Affective Computing and Intelligent Interaction, 116--122

    2019

  8. [16]

    Fragopanagos, N.; and Taylor, J. G. 2005. Emotion recognition in human--computer interaction. Neural Networks, 18(4): 389--405

    2005

  9. [17]

    Ghaleb, E.; Mertens, A.; Asteriadis, S.; and Weiss, G. 2021. Skeleton-based explainable bodily expressed emotion recognition through graph convolutional networks. In FG, 1--8

    2021

  10. [18]

    Glowinski, D.; Camurri, A.; Volpe, G.; Dael, N.; and Scherer, K. 2008. Technique for automatic emotion recognition by body gesture analysis. In CVPR Workshops, 1--6

    2008

  11. [19]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2021. Lora: Low-rank adaptation of large language models. arXiv:2106.09685

    2021 arXiv

  12. [20]

    Lee, J.; Lee, M.; Lee, D.; and Lee, S. 2023. Hierarchically Decomposed Graph Convolutional Networks for Skeleton-Based Action Recognition. In ICCV, 10444--10453

    2023

  13. [21]

    Li, C.; Wang, J.; Zhang, Y.; Zhu, K.; Hou, W.; Lian, J.; Luo, F.; Yang, Q.; and Xie, X. 2023 a . Large language models understand and can be enhanced by emotional stimuli. arXiv:2307.11760

    2023 arXiv

  14. [22]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023 b . Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, 19730--19742

    2023

  15. [23]

    Li, K.; He, Y.; Wang, Y.; Li, Y.; Wang, W.; Luo, P.; Wang, Y.; Wang, L.; and Qiao, Y. 2023 c . Videochat: Chat-centric video understanding. arXiv:2305.06355

    2023 arXiv

  16. [24]

    Li, K.; Wang, Y.; He, Y.; Li, Y.; Wang, Y.; Liu, Y.; Wang, Z.; Xu, J.; Chen, G.; Luo, P.; et al. 2024. Mvbench: A comprehensive multi-modal video understanding benchmark. In CVPR, 22195--22206

    2024

  17. [25]

    Li, S.; and Deng, W. 2022. Deep Facial Expression Recognition: A Survey. IEEE Transactions on Affective Computing, 13(3): 1195--1215

    2022

  18. [26]

    L.; De Lima Costa, W.; Mart \' nez, E

    Lima, M. L.; De Lima Costa, W.; Mart \' nez, E. T.; and Teichrieb, V. 2024. ST-Gait++: Leveraging spatio-temporal convolutions for gait-based emotion recognition on videos. In CVPR, 302--310

    2024

  19. [27]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023. Visual Instruction Tuning. In NeurIPS, volume 36, 34892--34916

    2023

  20. [28]

    Lu, H.; Hu, X.; and Hu, B. 2023. See Your Emotion from Gait Using Unlabeled Skeleton Data. In AAAI, volume 37, 1826--1834

    2023

  21. [29]

    Maaz, M.; Rasheed, H.; Khan, S.; and Khan, F. S. 2023. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv:2306.05424

    2023 arXiv

  22. [30]

    L.; Atkins, M

    Mandryk, R. L.; Atkins, M. S.; and Inkpen, K. M. 2006. A continuous and objective evaluation of emotional experience with interactive play environments. In SIGCHI, 1027--1036

    2006

  23. [31]

    M.; Dorbala, V

    Narayanan, V.; Manoghar, B. M.; Dorbala, V. S.; Manocha, D.; and Bera, A. 2020. Proxemo: Gait-based emotion learning and multi-view proxemic fusion for socially-aware robot navigation. In IROS, 8200--8207

    2020

  24. [32]

    A.; Kami \'n ska, D.; Sapi \'n ski, T.; Escalera, S.; and Anbarjafari, G

    Noroozi, F.; Corneanu, C. A.; Kami \'n ska, D.; Sapi \'n ski, T.; Escalera, S.; and Anbarjafari, G. 2018. Survey on emotional body gesture recognition. IEEE Transactions on Affective Computing, 12(2): 505--523

    2018

  25. [33]

    O g uz, A.; and Ertu g rul, \"O . F. 2024. Emotion recognition by skeleton-based spatial and temporal analysis. Expert Systems with Applications, 238: 121981

    2024

  26. [34]

    OpenAI. 2023. GPT-4: Technical Report. https://cdn.openai.com/papers/gpt-4.pdf. Accessed: 2024-08-15

    2023

  27. [35]

    OpenAI. 2024. ChatGPT: An AI Language Model. https://chat.openai.com/. Accessed: 2024-08-15

    2024

  28. [36]

    Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; et al. 2022. Training language models to follow instructions with human feedback. In NeurIPS, volume 35, 27730--27744

    2022

  29. [37]

    Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. 2019. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS, volume 32

    2019

  30. [38]

    Peng, Q.; Zheng, C.; and Chen, C. 2024. A Dual-Augmentor Framework for Domain Generalization in 3D Human Pose Estimation. In CVPR, 2240--2249

    2024

  31. [39]

    Piana, S.; Staglian \`o , A.; Odone, F.; and Camurri, A. 2016. Adaptive body gesture representation for automatic emotion recognition. ACM Transactions on Interactive Intelligent Systems, 6(1): 1--31

    2016

  32. [40]

    Qiu, F.; Zhang, W.; Liu, C.; Li, L.; Du, H.; Guo, T.; and Yu, X. 2024. Language-guided Multi-modal Emotional Mimicry Intensity Estimation. In CVPR, 4742--4751

    2024

  33. [41]

    Qu, H.; Cai, Y.; and Liu, J. 2024. Llms are good action recognizers. In CVPR, 18395--18406

    2024

  34. [42]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In ICML, 8748--8763

    2021

  35. [43]

    Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; Sutskever, I.; et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8): 9

    2019

  36. [44]

    Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140): 1--67

    2020

  37. [45]

    Research, G. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv:2403.05530

    2024 arXiv

  38. [46]

    Sapi \'n ski, T.; Kami \'n ska, D.; Pelikant, A.; Ozcinar, C.; Avots, E.; and Anbarjafari, G. 2019. Multimodal database of emotional speech, video and gestures. In ICPR, 153--163

    2019

  39. [47]

    Shi, L.; Zhang, Y.; Cheng, J.; and Lu, H. 2019. Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition. In CVPR, 12026--12035

    2019

  40. [48]

    Taori, R.; Gulrajani, I.; Zhang, T.; Dubois, Y.; Li, X.; Guestrin, C.; Liang, P.; and Hashimoto, T. B. 2023. Stanford Alpaca: An Instruction-following LLaMA model. https://github.com/tatsu-lab/stanford_alpaca

    2023

  41. [49]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient foundation language models. arXiv:2302.13971

    2023 arXiv

  42. [50]

    Wang, M.; Xing, J.; Mei, J.; Liu, Y.; and Jiang, Y. 2023. Actionclip: Adapting language-image pretrained models for video action recognition. IEEE Transactions on Neural Networks and Learning Systems, 1--13

    2023

  43. [51]

    Wang, T.; Liu, S.; He, F.; Dai, W.; Du, M.; Ke, Y.; and Ming, D. 2024. Emotion Recognition From Full-Body Motion Using Multiscale Spatio-Temporal Network. IEEE Transactions on Affective Computing, 15(3): 898--912

    2024

  44. [52]

    Xiang, W.; Li, C.; Zhou, Y.; Wang, B.; and Zhang, L. 2023. Generative action description prompts for skeleton-based action recognition. In ICCV, 10276--10285

    2023

  45. [53]

    Yan, S.; Xiong, Y.; and Lin, D. 2018. Spatial temporal graph convolutional networks for skeleton-based action recognition. In AAAI, volume 32, 7444--7452

    2018

  46. [54]

    You, Z.; Wen, Z.; Chen, Y.; Li, X.; Zeng, R.; Wang, Y.; and Tan, M. 2024. Toward Long Video Understanding via Fine-detailed Video Story Generation. IEEE Transactions on Circuits and Systems for Video Technology, 1--1

    2024

  47. [55]

    Zeng, R.; Zhuo, Y.; Li, J.; Yang, Y.; Wu, H.; Chen, Q.; Hu, X.; and Leung, V. C. 2024. Improving Video Moment Retrieval by Auxiliary Moment-Query Pairs with Hyper-Interaction. IEEE Transactions on Circuits and Systems for Video Technology, 1--1

    2024

  48. [56]

    Zhai, Y.; Jia, G.; Lai, Y.-K.; Zhang, J.; Yang, J.; and Tao, D. 2024. Looking into Gait for Perceiving Emotions via Bilateral Posture and Movement Graph Convolutional Networks. IEEE Transactions on Affective Computing, 1--15

    2024

  49. [57]

    Zhang, H.; Yi, P.; Liu, R.; and Zhou, D. 2021. Emotion recognition from body movements with as-lstm. In ICVR, 26--32

    2021

  50. [58]

    Zhang, M.; Yu, L.; Zhang, K.; Du, B.; Zhan, B.; Chen, S.; Jiang, X.; Guo, S.; Zhao, J.; Wang, Y.; et al. 2020. Kinematic dataset of actors expressing emotions. Scientific data, 7(1): 292

    2020

  51. [59]

    Zhang, Z. 2012. Microsoft kinect sensor and its effect. IEEE multimedia, 19(2): 4--10

    2012

  52. [60]

    Zheng, C.; Wu, W.; Chen, C.; Yang, T.; Zhu, S.; Shen, J.; Kehtarnavaz, N.; and Shah, M. 2023. Deep learning-based human pose estimation: A survey. ACM Computing Surveys

    2023

  53. [61]

    Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv:2304.10592

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.