pith. sign in

arxiv: 2604.16629 · v1 · submitted 2026-04-17 · 💻 cs.CV · cs.GR

Amortized Inverse Kinematics via Graph Attention for Real-Time Human Avatar Animation

Muhammad Saif Ullah Khan , Chen-Yu Wang , Tim Prokosch , Michael Lorenz , Bertram Taetz , Didier Stricker This is my paper

Pith reviewed 2026-05-10 08:50 UTC · model grok-4.3

classification 💻 cs.CV cs.GR
keywords jointik-gatrotationsanimationavatarkinematicorientationspose
0
0 comments X

The pith

A lightweight graph attention model amortizes inverse kinematics to produce animation-ready rotations from sparse joint positions at over 650 FPS on CPU.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Animation systems need joint rotations to move a 3D character, but many tracking systems only provide the 3D positions of a few joints. Recovering the missing rotation angles is underconstrained because bones can twist around their long axis without changing the end position. Classical solvers fix this by running iterative optimization that is slow and can fail on noisy data. The new method treats the skeleton as a graph where joints are nodes and bones are edges. A graph attention network passes messages along these connections to infer the missing rotations. It predicts rotations in a special bone-aligned frame that makes the twist direction explicit and easy to convert back to standard local rotations. The network is small, uses a 6D rotation representation, and is trained with a loss that measures angular error on the sphere. An optional term checks that the predicted rotations produce the original positions when run through forward kinematics.

Core claim

With 374K parameters and over 650 FPS on CPU, IK-GAT outperforms VPoser-based per-frame iterative optimization without warm-start at significantly lower cost, and is robust to initial pose and input noise.

Load-bearing premise

The bone-aligned world-frame rotation representation plus the known kinematic tree and rest pose are sufficient to resolve orientation ambiguities (especially twist) from position inputs alone.

Figures

Figures reproduced from arXiv: 2604.16629 by Bertram Taetz, Chen-Yu Wang, Didier Stricker, Michael Lorenz, Muhammad Saif Ullah Khan, Tim Prokosch.

Figure 1
Figure 1. Figure 1: Amortized inverse kinematics. Given 3D joint positions, IK-GAT predicts per-joint rotations in bone-aligned world space and analytically recovers standard local rotations, enabling direct animation or use in body models. Abstract Inverse kinematics (IK) is a core operation in animation, robotics, and biomechanics: given Cartesian constraints, recover joint rotations under a known kinematic tree. In many re… view at source ↗
Figure 2
Figure 2. Figure 2: Twist ambiguity problem. Identical joint positions (blue dots) are produced by any rotation around the bone axis (red). This makes the inverse problem ill-posed without strong anatomical pri￾ors. Neural IK methods attempt to learn these priors from data. 3.2.1. Rest-pose bone frames (precomputed once per rig) Let u be a fixed unit vector denoting the global up direction. For each joint i in a given rig wit… view at source ↗
Figure 3
Figure 3. Figure 3: IK-GAT architecture. The model accepts 3D joint positions as input, which are projected and augmented with learnable posi￾tional embeddings. The core encoder leverages a Kinematic Prior, defined by the skeletal parent-child relationships, to structure the Graph Attention (GAT) layers. The network employs a dual-residual design, featuring both local skip connections within GAT blocks and a global shortcut f… view at source ↗
Figure 4
Figure 4. Figure 4: Per-joint improvement over the strongest dense base￾line. Gains concentrate on distal and twist-sensitive joints such as ankles, feet, elbows, neck, and head, showing where kinematic mes￾sage passing and distal local refinement matter most. Forearms re￾main hard because they have minimal positional evidence for twist [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Ablation of graph structure and attention. Bidirectional kinematic message passing with attention is critical. Fully con￾nected graphs dilute the anatomical inductive bias, one-way graphs block corrective feedback along the chain, and non-attentive graph propagation cannot adapt neighbor importance to the current pose. Effect of IK-GAT components. Tab 3 shows that positional em￾beddings are indispensable: … view at source ↗
Figure 5
Figure 5. Figure 5: Iteration budget versus accuracy on AMASS. Optimiza￾tion baselines need tens to hundreds of iterations to approach the accuracy reached by a single forward pass of IK-GAT. Runtime. Single-frame inference reaches 693 FPS on CPU and 448 FPS on GPU (RTX 4060), while GPU provides up to 3.7× higher throughput under batching (Appendix A). 4.2.2. Ablation study Effect of bone-aligned world rotation space. Trainin… view at source ↗
Figure 9
Figure 9. Figure 9: Real-time deployment setting. A tracked human motion is first converted into a dense stream of 3D joints by an upstream markerless multi-view system, then mapped by IK-GAT to per-joint rotations in bone-aligned world space, analytically recovered to UE5-Mannequin local rotations, and rendered directly in Unreal Engine. This benchmark evaluates the complete animation-facing inverse-kinematics pipeline rathe… view at source ↗
Figure 7
Figure 7. Figure 7: Effect of FK-consistency weight α. A small FK regular￾ization improves both local-rotation accuracy and downstream ge￾ometry by tying the predicted rotations back to the observed joints. Larger weights over-constrain training and make the optimization trade-off increasingly unstable, so we use α = 0.1 throughout. regime. Scaling from Tiny to Base (default) yields the expected im￾provement, but larger model… view at source ↗
Figure 8
Figure 8. Figure 8: Model scaling. Performance improves rapidly from Tiny to Base, then largely saturates. This indicates that the main gains come from the representation and the anatomical inductive bias rather than from scaling capacity alone [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: UE-IK acquisition process. Real motion is captured with Xsens IMUs, streamed to UE, and exported as paired joint po￾sitions and rotations. Marketplace animations are exported through the same pipeline, giving unified training data for the benchmark [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗
read the original abstract

Inverse kinematics (IK) is a core operation in animation, robotics, and biomechanics: given Cartesian constraints, recover joint rotations under a known kinematic tree. In many real-time human avatar pipelines, the available signal per frame is a sparse set of tracked 3D joint positions, whereas animation systems require joint orientations to drive skinning. Recovering full orientations from positions is underconstrained, most notably because twist about bone axes is ambiguous, and classical IK solvers typically rely on iterative optimization that can be slow and sensitive to noisy inputs. We introduce IK-GAT, a lightweight graph-attention network that reconstructs full-body joint orientations from 3D joint positions in a single forward pass. The model performs message passing over the skeletal parent-child graph to exploit kinematic structure during rotation inference. To simplify learning, IK-GAT predicts rotations in a bone-aligned world-frame representation anchored to rest-pose bone frames. This parameterization makes the twist axis explicit and is exactly invertible to standard parent-relative local rotations given the kinematic tree and rest pose. The network uses a continuous 6D rotation representation and is trained with a geodesic loss on SO(3) together with an optional forward-kinematics consistency regularizer. IK-GAT produces animation-ready local rotations that can directly drive a rigged avatar or be converted to pose parameters of SMPL-like body models for real-time and online applications. With 374K parameters and over 650 FPS on CPU, IK-GAT outperforms VPoser-based per-frame iterative optimization without warm-start at significantly lower cost, and is robust to initial pose and input noise

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The paper introduces IK-GAT, a lightweight graph-attention network for amortized inverse kinematics that reconstructs full-body joint orientations from sparse 3D joint positions in a single forward pass. It uses a bone-aligned world-frame rotation representation (invertible to local rotations given the kinematic tree and rest pose), a continuous 6D rotation encoding, geodesic loss on SO(3), and an optional forward-kinematics consistency regularizer. The model has 374K parameters, runs at >650 FPS on CPU, and is claimed to outperform VPoser-based per-frame iterative optimization without warm-start while being robust to initial pose and input noise.

Significance. If the speed, accuracy, and robustness claims hold, the work provides a practical amortized alternative to iterative IK solvers for real-time avatar animation pipelines. The graph-attention exploitation of the skeletal tree and the explicit twist-axis parameterization are clear strengths that could extend to other kinematic inference tasks in animation and robotics. The low parameter count and CPU speed are particularly enabling for online applications.

major comments (1)
  1. [Training objective and loss formulation] The bone-aligned world-frame output makes twist explicit, but joint positions are invariant to rotation about each bone axis. The optional forward-kinematics consistency regularizer (described in the training section) only penalizes mismatches in reconstructed joint positions and therefore supplies zero gradient on the twist DOFs. All twist inference thus flows exclusively from the supervised geodesic loss on ground-truth orientations, making the robustness-to-noise and generalization claims rest on the assumption that the training distribution adequately covers relevant twist variations—an assumption that requires explicit validation via targeted ablations or twist-specific test sets.
minor comments (2)
  1. [Abstract] The abstract states performance and robustness claims but provides no quantitative metrics, dataset names, or ablation details; these should be summarized with key numbers even in the abstract for clarity.
  2. [Method] The exact conversion procedure from the predicted bone-aligned 6D representation back to standard parent-relative local rotations (given rest pose) could be stated as a short algorithm or set of equations to aid reproducibility.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No explicit free parameters, axioms, or invented entities are stated in the abstract; the approach is data-driven.

pith-pipeline@v0.9.0 · 5607 in / 1000 out tokens · 62846 ms · 2026-05-10T08:50:31.193447+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages

  1. [1]

    Inverse kinemat- ics: a review of existing techniques and introduction of a new fast itera- tive solver

    [AL09] ARISTIDOU, ANDREASand LASENBY, JOAN. “Inverse kinemat- ics: a review of existing techniques and introduction of a new fast itera- tive solver”. (2009)

    work page 2009

  2. [2]

    FABRIK: A Fast, Iterative Solver for the Inverse Kinematics Problem

    [AL11] ARISTIDOU, ANDREASand LASENBY, JOAN. “FABRIK: A Fast, Iterative Solver for the Inverse Kinematics Problem”.Graphical Models 73.5 (2011), 243–260.DOI:10.1016/j.gmod.2011.05.0032. [AWS*22] AKADA, HIROYASU, WANG, JIAN, SHIMADA, SOSHI, et al. “Unrealego: A new dataset for robust egocentric 3d human motion cap- ture”.European Conference on Computer Visi...

    work page doi:10.1016/j.gmod.2011.05.0032 2011

  3. [3]

    Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot

    [BAG*24] BARADEL, FABIEN, ARMANDO, MATTHIEU, GALAAOUI, SALMA, et al. “Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot”.European Conference on Computer Vision (ECCV). 2024

    work page 2024

  4. [4]

    2022, 1787–1797

    Pro- ceedings of Machine Learning Research. 2022, 1787–1797

    work page 2022

  5. [5]

    Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image

    [BKL*16] BOGO, FEDERICA, KANAZAWA, ANGJOO, LASSNER, CHRISTOPH, et al. “Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image”.European Conference on Computer Vision (ECCV). 2016

    work page 2016

  6. [6]

    [Bus04] BUSS, SAMUELR.Introduction to Inverse Kinematics with Ja- cobian Transpose, Pseudoinverse and Damped Least Squares Methods. Tech. rep. Technical report; made available online by the author. Univer- sity of California, San Diego, Apr. 2004

    work page 2004

  7. [7]

    Embodied Social Experiences in Hybrid Shared Spaces

    [CGdB*23] CORAGGIO, MARCO, GROTTA, ANTONIO, di BERNARDO, MARIO, et al. “Embodied Social Experiences in Hybrid Shared Spaces”. (2023)

    work page 2023

  8. [8]

    OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement

    [DAA*07] DELP, SCOTTL., ANDERSON, FRANKC., ARNOLD, ALLI- SONS., et al. “OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement”.IEEE Transactions on Biomedical Engineering54.11 (2007), 1940–1950

    work page 2007

  9. [9]

    [Gho*19] GHORBANI, NIMAet al.human_body_prior: VPoser and optimization-based body priors (software).https://github.com/ nghorbani/human_body_prior. 2019 2,

    work page 2019

  10. [10]

    Retargetting motion to new characters

    [Gle98] GLEICHER, MICHAEL. “Retargetting motion to new characters”. Proceedings of the 25th annual conference on Computer graphics and interactive techniques. 1998, 33–42

    work page 1998

  11. [11]

    KinePose: A temporally optimized inverse kine- matics technique for 6DOF human pose estimation with biomechani- cal constraints

    [GMB*22] GILDEA, KEVIN, MERCADAL-BAUDART, CLARA, BLYTH- MAN, RICHARD, et al. “KinePose: A temporally optimized inverse kine- matics technique for 6DOF human pose estimation with biomechani- cal constraints”.24th Irish Machine Vision and Image Processing Con- ference. Irish Pattern Recognition & Classification Society. 2022, 105– 112

    work page 2022

  12. [12]

    Humans in 4D: Reconstructing and Tracking Hu- mans with Transformers

    [GPR*23] GOEL, SHUBHAM, PAVLAKOS, GEORGIOS, RAJASEGARAN, JATHUSHAN, et al. “Humans in 4D: Reconstructing and Tracking Hu- mans with Transformers”.Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2023

    work page 2023

  13. [13]

    Reuse of motion capture data in animation: A review

    [GY03] GENG, WEIDONGand YU, GINO. “Reuse of motion capture data in animation: A review”.International Conference on Computational Science and Its Applications. Springer. 2003, 620–629

    work page 2003

  14. [14]

    Deep inertial poser: Learning to reconstruct human pose from sparse inertial measurements in real time

    [HKA*18] HUANG, YINGHAO, KAUFMANN, MANUEL, AKSAN, EMRE, et al. “Deep inertial poser: Learning to reconstruct human pose from sparse inertial measurements in real time”.ACM Transactions on Graph- ics (TOG)37.6 (2018), 1–15

    work page 2018

  15. [15]

    Metrics for 3D Rotations: Comparison and Analysis

    [Huy09] HUYNH, DUQ. “Metrics for 3D Rotations: Comparison and Analysis”.Journal of Mathematical Imaging and Vision35.2 (2009), 155–164.DOI:10.1007/s10851-009-0161-23,

    work page doi:10.1007/s10851-009-0161-23 2009

  16. [16]

    MANIKIN: Neural Inverse Kinematics from Sparse Signals

    [JCZ*24] JIANG, ZIANG, CHEN, ZHIPENG, ZHOU, YUXUAN, et al. “MANIKIN: Neural Inverse Kinematics from Sparse Signals”.Euro- pean Conference on Computer Vision (ECCV). Preprint/Camera-ready may differ. 2024 2, 3,

    work page 2024

  17. [17]

    End-to-end Recovery of Human Shape and Pose

    [KBJM18] KANAZAWA, ANGJOO, BLACK, MICHAELJ., JACOBS, DAVIDW., and MALIK, JITENDRA. “End-to-end Recovery of Human Shape and Pose”.Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR). 2018

    work page 2018

  18. [18]

    Skinning with dual quaternions

    [KCŽO07] KAVAN, LADISLAV, COLLINS, STEVEN, ŽÁRA, JI ˇRÍ, and O’SULLIVAN, CAROL. “Skinning with dual quaternions”.Proceedings of the 2007 symposium on Interactive 3D graphics and games. 2007, 39– 46

    work page 2007

  19. [19]

    SIMSPINE: A Biomechanics-Aware Simulation Framework for 3D Spine Motion Annotation and Benchmarking

    [KS26] KHAN, MUHAMMADSAIFULLAHand STRICKER, DIDIER. “SIMSPINE: A Biomechanics-Aware Simulation Framework for 3D Spine Motion Annotation and Benchmarking”.arXiv preprint arXiv:2602.20792(2026)

    work page arXiv 2026

  20. [20]

    NIKI: Neural In- verse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation

    [LBL*23] LI, JIEFENG, BIAN, SIYUAN, LIU, QI, et al. “NIKI: Neural In- verse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation”.IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR). 2023

    work page 2023

  21. [21]

    En- hanced dual quaternion skinning for production use

    [LLS*13] LEE, GENES, LIN, ANDY, SCHILLER, MATT, et al. “En- hanced dual quaternion skinning for production use.”SIGGRAPH Talks. 2013, 9–1

    work page 2013

  22. [22]

    SMPL: A Skinned Multi-Person Linear Model

    [LMR*15] LOPER, MATTHEW, MAHMOOD, NAUREEN, ROMERO, JAVIER, et al. “SMPL: A Skinned Multi-Person Linear Model”.ACM Transactions on Graphics34.6 (2015) 2,

    work page 2015

  23. [23]

    HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation

    [LXC*21] LI, JIEFENG, XU, CHAO, CHEN, ZHICUN, et al. “HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation”.IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2021 2, 3,

    work page 2021

  24. [24]

    Morgan kaufmann, 2000

    [Men00] MENACHE, ALBERTO.Understanding motion capture for com- puter animation and video games. Morgan kaufmann, 2000

    work page 2000

  25. [25]

    AMASS: Archive of Motion Capture As Surface Shapes

    [MGT*19] MAHMOOD, NAUREEN, GHORBANI, NIMA, TROJE, NIKO- LAUSF., et al. “AMASS: Archive of Motion Capture As Surface Shapes”.Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2019

    work page 2019

  26. [26]

    Expressive Body Capture: 3D Hands, Face, and Body from a Single Image

    [PCG*19] PAVLAKOS, GEORGIOS, CHOUTAS, VASILEIOS, GHORBANI, NIMA, et al. “Expressive Body Capture: 3D Hands, Face, and Body from a Single Image”.IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR). 2019

    work page 2019

  27. [27]

    The Kinematics of Manipulators under Computer Control

    12 [Pie68] PIEPER, DONALDLEE. “The Kinematics of Manipulators under Computer Control”. PhD thesis. Stanford University, 1968

    work page 1968

  28. [28]

    SparsePoser: Real-Time Full-Body Motion Reconstruc- tion from Sparse Data

    [PY A*23] PONTON, ENRICO, YUN, SUNG-KYU, ARISTIDOU, AN- DREAS, et al. “SparsePoser: Real-Time Full-Body Motion Reconstruc- tion from Sparse Data”.ACM Transactions on Graphics(2023). To ap- pear/issue details vary by venue listing

    work page 2023

  29. [29]

    Skin- ning techniques for articulated deformable characters

    [RF16] RUMMAN, NADINEABUand FRATARCANGELI, MARCO. “Skin- ning techniques for articulated deformable characters”.Computer Graphics Theory and Applications(2016)

    work page 2016

  30. [30]

    et al.Quater-GCN: Enhancing 3D Human Pose Estimation with Orientation and Semi-supervised Training

    [Son*24] SONG, H. et al.Quater-GCN: Enhancing 3D Human Pose Estimation with Orientation and Semi-supervised Training. arXiv:2404.19279. 2024 7,

    work page arXiv 2024

  31. [31]

    JointTracker: Real-time inertial kinematic chain tracking with joint position estimation

    [TLM*25] TAETZ, BERTRAM, LORENZ, MICHAEL, MIEZAL, MARKUS, et al. “JointTracker: Real-time inertial kinematic chain tracking with joint position estimation”.Open Research Europe4 (2025), 33

    work page 2025

  32. [32]

    Least-squares estimation of transformation parameters between two point patterns

    [Ume02] UMEYAMA, SHINJI. “Least-squares estimation of transformation parameters between two point patterns”.IEEE Transactions on pattern analysis and machine intelligence13.4 (2002), 376–380

    work page 2002

  33. [33]

    Graph Attention Networks

    [VCC*18] VELI ˇCKOVI ´C, PETAR, CUCURULL, GUILLEM, CASANOVA, ARANTXA, et al. “Graph Attention Networks”.International Conference on Learning Representations (ICLR). 2018 3,

    work page 2018

  34. [34]

    Diffusionposer: Real-time human motion reconstruction from ar- bitrary sparse sensors using autoregressive diffusion

    [VLF*24] VANWOUWE, TOM, LEE, SEUNGHWAN, FALISSE, ANTOINE, et al. “Diffusionposer: Real-time human motion reconstruction from ar- bitrary sparse sensors using autoregressive diffusion”.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, 2513–2523

    work page 2024

  35. [35]

    com / wangsen1312 / joints2smpl

    [Wan*20] WANG, SENet al.joints2smpl: SMPL-like optimization from 3D joints (software).https : / / github . com / wangsen1312 / joints2smpl. 2020

    work page 2020

  36. [36]

    A Com- bined Optimization Method for Solving the Inverse Kinematics Problem of Mechanical Manipulators

    [WC91] WANG, LI-CHUNTOMMYand CHEN, CHIHCHENG. “A Com- bined Optimization Method for Solving the Inverse Kinematics Problem of Mechanical Manipulators”.IEEE Transactions on Robotics and Au- tomation7.4 (1991), 489–499

    work page 1991

  37. [37]

    Spa- tial Temporal Graph Convolutional Networks for Skeleton-Based Ac- tion Recognition

    [YXL18] YAN, SIJIE, XIONG, YUANJUN, and LIN, DAHUA. “Spa- tial Temporal Graph Convolutional Networks for Skeleton-Based Ac- tion Recognition”.AAAI Conference on Artificial Intelligence (AAAI). 2018

    work page 2018

  38. [38]

    HierarIK: Hier- archical Inverse Kinematics Solver for Human Body and Hand Pose Es- timation

    [YZX21a] YI, XINYU, ZHOU, YUXIAO, and XU, FENG. “HierarIK: Hier- archical Inverse Kinematics Solver for Human Body and Hand Pose Es- timation”.CAAI International Conference on Artificial Intelligence (CI- CAI), Lecture Notes in Computer Science. Springer; online date listed as 2022 in publisher metadata. 2021 2, 3,

    work page 2022

  39. [39]

    Transpose: Real-time 3d human translation and pose estimation with six inertial sen- sors

    [YZX21b] YI, XINYU, ZHOU, YUXIAO, and XU, FENG. “Transpose: Real-time 3d human translation and pose estimation with six inertial sen- sors”.ACM Transactions On Graphics (TOG)40.4 (2021), 1–13

    work page 2021

  40. [40]

    On the Continuity of Rotation Representations in Neural Networks

    [ZBL*19] ZHOU, YI, BARNES, CONNELLY, LU, JINGWAN, et al. “On the Continuity of Rotation Representations in Neural Networks”.Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019 2–4. [ZPT*19] ZHAO, LONG, PENG, XI, TIAN, YU, et al. “Semantic Graph Convolutional Networks for 3D Human Pose Regression”.Proceedings of...

    work page 2019