Pith. sign in

REVIEW 3 major objections 6 minor 41 references

A single multimodal model can estimate 3D pose from video and generate future or missing skeleton motion using one shared, vision-grounded token vocabulary.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 05:22 UTC pith:QREYG6YZ

load-bearing objection A genuinely new unified motion MLLM architecture with real generalization claims, but internal arithmetic errors and missing cross-modal token-agreement evidence mean the numbers need referee scrutiny before being trusted. the 3 major comments →

arxiv 2602.02401 v2 pith:QREYG6YZ submitted 2026-02-02 cs.CV

Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation

Xinshun Wang , Peiming Li , Ziyi Wang , Zhongbin Fang , Zhichao Deng , Songtao Wu , Jason Li , Mengyuan Liu This is my paper
classification cs.CV
keywords 3D pose estimationmotion predictionmotion in-betweeningmultimodal large language modelvector quantized autoencoderhybrid codebookskeleton motion generationvision-language model
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Superman tries to show that perception and generation of human motion are not separate problems: a single multimodal large language model, given video, text, and skeleton tokens, can estimate 3D poses from video, predict future motion from past skeletons, and fill in missing frames between keyframes. The load-bearing idea is a Vision-Guided Motion Tokenizer that builds a discrete motion vocabulary where every token is anchored both to visual appearance and to 3D skeleton geometry, so reading motion and writing motion share one language. On standard benchmarks the model reports state-of-the-art or competitive errors on all three tasks, and it transfers to an unseen dataset after training on one dataset alone. If the claim holds, the fragmented pipeline of separate pose estimators and motion generators can be replaced by one generative model.

Core claim

The paper's central claim is that motion can be treated as a universal discrete language, and that a single decoder-only multimodal large language model trained on that language can unite two camps that have been separate: models that only perceive (video to text) and models that only generate (skeleton to skeleton). The discovery is the Vision-Guided Motion Tokenizer: unlike prior motion vocabularies built from skeletons alone, it trains a VQ-VAE with a hybrid codebook in which each code is a paired visual prototype and geometric prototype. Quantization selects a token by minimizing the summed distance of the current visual and skeletal features to the pair, and the model then reconstructs

What carries the argument

Vision-Guided Motion Tokenizer (VGMT): a vector-quantized autoencoder that converts a 16-frame motion window into discrete tokens. Its hybrid codebook is the central object — each of 8,192 entries is a pair (visual prototype, skeleton prototype) — and Eq. 2 chooses the token that minimizes the joint Euclidean distance of the visual and skeleton features to the pair. A Visual-Skeleton Attention (VSA) module samples visual features adaptively around projected joints, and an optional Motion-Aware Fine-Tuning (MAFT) module injects skeletal geometry into the LLM's visual stream via cross-attention, adding under 0.2% parameters. The MLLM autoregressively predicts the token sequence, with pose esti

Load-bearing premise

The method assumes that minimizing the summed distance of visual and skeleton features to a paired code is enough to keep the two modalities aligned on the same motion; if the visual and skeleton halves of the codebook drift apart, the claimed unified motion language is two separate vocabularies.

What would settle it

Run the tokenizer twice on the same motion window — once with the raw video as input and once with the paired 3D skeleton — and check whether both produce the same token index. If cross-modal token agreement is barely above chance on the test set, the hybrid codebook is not serving as a shared language, and the pose-estimation and generation branches are effectively using different vocabularies.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the central claim is correct, motion perception and motion generation no longer need separate model families: one parameter-efficient fine-tuned LLM covers video-to-pose, pose-to-future, and keyframe interpolation.
  • Joint training on all three tasks outperforms training on each task separately, so the shared token vocabulary transfers knowledge across perception and generation.
  • Grounding the motion vocabulary in vision improves even skeleton-only generation tasks, meaning visual features encode motion cues that purely geometric skeletons miss.
  • The unified model generalizes zero-shot from one dataset to an unseen one, with the tokenizer retaining high code utilization, implying the learned vocabulary is not dataset-specific.
  • Scaling the LLM and the codebook both reduce pose error, so the architecture has a clear path to further gains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same frozen tokenizer could be attached to text-conditioned generation, since the visual prototype half of each code gives text a natural anchor to the motion vocabulary; the paper trains on video/skeleton inputs and does not test text prompts, so this is a natural next check.
  • Editorial inference: the strong zero-shot results on unseen data may owe more to the discrete cross-modal vocabulary than to the LLM itself; a direct comparison against a continuous-regression model with the same visual front end would isolate that contribution.
  • Editorial inference: the hybrid-codebook pairing should be probed for cross-modal consistency; if the same motion video and its paired skeleton yield different token indices frequently, the model is likely relying on task-specific shortcuts rather than one shared motion language.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Superman, a single MLLM-based framework that performs 3D pose estimation from video, skeleton-based motion prediction, and motion in-betweening. The key novelty is a Vision-Guided Motion Tokenizer (VGMT), a VQ-VAE with a hybrid codebook of paired visual and geometric prototypes, intended to create a unified cross-modal motion vocabulary. The MLLM is then trained to autoregressively emit skeleton tokens conditioned on video, text, or skeleton inputs. Experiments on Human3.6M and 3DPW report state-of-the-art or competitive MPJPE/N-MPJPE on all three tasks, together with ablations and visualizations.

Significance. If the central claims hold, the paper would be a meaningful step toward unifying motion perception and generation in one architecture: a single model performing video-to-3D-skeleton estimation, skeleton motion prediction, and in-betweening, with a visually grounded discrete motion vocabulary and strong zero-shot generalization from Human3.6M to 3DPW. The overall architecture is plausible, and the paper includes useful ablations (VSA, MAFT, fusion weights, multi-task training) and efficiency analyses. The main weakness is that the core cross-modal vocabulary claim is not directly evidenced: the training objective and the reported analyses do not show that video-only and skeleton-only inputs produce the same tokens for the same motion. There are also several internal numerical and dimensional inconsistencies that currently undermine the empirical case.

major comments (3)
  1. [Sec. 3.1, Eqs. (2)-(3); Fig. 7; Table 8] The paper's central novelty is that VGMT creates a unified cross-modal motion vocabulary, but no experiment directly tests cross-modal consistency. In Eq. (3), the reconstruction loss uses only the skeletal code c_s; the visual code c_v is never decoded. The only coupling between the visual and skeletal branches is the shared argmin in Eq. (2) and the two commitment losses. There is no explicit loss aligning c_v_k with c_s_k, nor any reported metric for whether the same motion window yields the same token from video-only and skeleton-only inputs. Codebook utilization (Fig. 7) and the 'semantic sphere' visualization (Fig. S13) do not address this. Table 8 shows that joint training helps downstream pose estimation, but it does not show that the two modalities share one vocabulary. Please add either an explicit cross-modal alignment loss or a quantitative cross-modal retrieval/token-agreeme
  2. [Sec. 4, Implementation Details] There is a dimensionality mismatch in the description of VGMT. It states that 'Both skeletal and visual features have 1024 dimensions, half of the dimensions of the codebook' while 'The Hybrid Codebook contains K=8192 codes, with a dimension of D=2048.' Then Eq. (2) evaluates ||z_v_w - c_v_k|| and ||z_s_w - c_s_k||, which requires the feature vectors and code vectors to have the same dimensionality. As written, 1024-dimensional features cannot be compared with 2048-dimensional code vectors. A projection layer or a different codebook dimension must be specified. Because Eq. (2) is the core of the tokenizer, this is essential for reproducibility.
  3. [Tables 2 and 4] The reported 'Avg' values for MP and MIB are not consistent with the per-horizon columns if 'Avg' is their mean. For example, in Table 2, MotionBERT MP Avg is 29.94 while (18.67+26.82+50.33)/3 = 31.94; its MIB Avg is 42.37, which is below both listed MIB columns 44.86 and 53.16. The same pattern occurs for most rows, including Superman (MP Avg 26.13 vs 27.30; MIB Avg 30.61 vs 35.56). Table 4 shows similar discrepancies. If 'Avg' is a different aggregate (e.g., averaged over all test frames), that definition must be stated explicitly; if it is meant as the mean of the listed columns, the numbers must be corrected. The SOTA claims rely on these aggregates, so this needs to be fixed before the results can be accepted.
minor comments (6)
  1. [Sec. 1] Typo: '3PDW' should be '3DPW'.
  2. [Fig. 6 caption] The caption says 'SiC [19]' but Skeleton-in-Context is reference [30]; Human-in-Context is [19]. Please correct the citation.
  3. [Sec. 4.1 and Table 5] The claim that MAFT adds '<0.2% extra parameters' is inconsistent with Table 5, which reports MAFT parameters of 25.45M against a total of 9605M, i.e., about 0.27%. Please reconcile the percentage.
  4. [Tables 2 and 4] Please clarify the training protocol for the published baselines used in Tables 2 and 4. The text states that all models are trained only on Human3.6M, but it is not stated whether these models were retrained for this comparison or whether published checkpoints (possibly trained on other data) were used. Since the tables are used to support zero-shot and SOTA claims, this should be explicit.
  5. [Sec. 4.3 and Table 8] The notation 'Fused: β_s, β_v = 0.3,0.7' is ambiguous; it should be written as '(β_s=0.3, β_v=0.7)' for clarity.
  6. [Sec. 3.2] For skeleton-only tasks, the paper does not specify how the frozen VGMT tokenizes an input when only skeleton features are available. Eq. (2) uses both z_v and z_s; the inference-time selection rule for MP/MIB must be stated.

Circularity Check

0 steps flagged

No significant circularity: benchmark-driven results; self-cited baselines are not load-bearing.

full rationale

The paper's central claims are empirical and benchmark-driven: the Vision-Guided Motion Tokenizer and the unified MLLM are evaluated by held-out MPJPE on Human3.6M and 3DPW. Eq. 2 defines the hybrid quantization rule and Eq. 3 defines the training objective; neither equation defines the reported evaluation metrics, and the reported numbers are not refits of the target quantities. The 'unified cross-modal vocabulary' claim is an architectural assertion supported by ablations (Tables 6-8) and codebook visualizations, not a derived theorem; the absence of an explicit cross-modal alignment loss is a validity/evidence gap, not a circularity, because the vocabulary claim is not the same as the MPJPE prediction being made. The references to HiC [19] and SiC [30] are from overlapping author groups, but they are used only as baseline comparisons and do not justify the VGMT or the LLM design; no load-bearing argument reduces to those citations. No step in the paper's derivation chain is equivalent to its inputs by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 1 invented entities

The central claim rests on standard ML engineering assumptions plus one ad hoc modeling assumption: that a simple joint-distance quantization rule creates genuine visual-skeleton alignment. The main free parameters are hyperparameters selected on the target benchmark. The paper introduces no new physical entity, but the hybrid codebook is a new representational construct with no independent evidence outside its own experiments.

free parameters (4)
  • Commitment weights beta_s, beta_v = 0.5, 0.5
    Eq. 3; Table 8 sweeps beta_s,beta_v and picks 0.5 on Human3.6M to minimize reconstruction error and N-MPJPE. These values are chosen by ablation on the target benchmark.
  • Codebook capacity K x D = 8192 x 2048
    Implementation Details, Section 4. The scaling analysis in Fig. 8 motivates larger codebooks, but the exact capacity is a design choice tuned on the target benchmark.
  • LoRA rank r = 8
    Implementation Details, Section 4. Chosen by design for Qwen2.5-VL-7B adaptation; no ablation is reported.
  • MAFT hidden dimension and heads = 1280, 8 heads
    Implementation Details, Section 3.2/4. Set to match the Qwen2.5-VL vision embedding dimension; no ablation is reported.
axioms (6)
  • domain assumption Pretrained Qwen2.5-VL-7B can be adapted via LoRA to autoregressively emit discrete motion tokens and maintain temporal coherence.
    Section 3.2 relies on Qwen2.5-VL-7B as the sequence processor. No guarantee is given that its text/video pretraining transfers to skeleton-token sequences; this is an empirical assumption.
  • domain assumption Human3.6M paired video and skeleton data is sufficient to learn a universal cross-modal motion vocabulary that transfers to 3DPW.
    VGMT and the MLLM are trained only on Human3.6M; Table 4 claims zero-shot transfer to 3DPW without external evidence of vocabulary coverage.
  • domain assumption 2D joint projections from an off-the-shelf 2D pose estimator provide reliable reference points for VSA/MAFT, and residual errors can be corrected by learned offsets.
    Section 3.1 Eq. 1 and Supplement Figures S5/S6 demonstrate correction qualitatively, but the quantitative reliability of the 2D estimator is not analyzed.
  • ad hoc to paper Minimizing the joint Euclidean distance in Eq. 2 aligns visual and skeletal prototypes without an explicit cross-modal alignment loss.
    The paper does not train a contrastive or reconstruction alignment between cv_k and cs_k. The hybrid codebook alignment is asserted rather than derived or separately validated.
  • standard math VQ-VAE with stop-gradient commitment loss (Eq. 3) is a valid path to a discrete motion vocabulary.
    Standard VQ-VAE objective from Van Den Oord et al. [26], used as-is.
  • domain assumption Converting 3DPW SMPL vertices to the Human3.6M skeleton format via a pre-trained matrix preserves evaluation-relevant geometry.
    Data Preprocessing, Section 4. The conversion error is not quantified.
invented entities (1)
  • Hybrid codebook of paired visual and geometric prototypes no independent evidence
    purpose: Creates the cross-modal motion vocabulary by pairing each token with a visual prototype and a skeleton prototype, quantizing both modalities with one index.
    The alignment between the paired prototypes is validated only on the paper's own Human3.6M/3DPW experiments; no external falsifiable handle is provided beyond the benchmark numbers themselves.

pith-pipeline@v1.3.0-alltime-deepseek · 19665 in / 20111 out tokens · 184303 ms · 2026-08-03T05:22:54.332754+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation." pith.science (2026). https://pith.science/paper/QREYG6YZ

@misc{pith2026260202401,
  author       = {Pith},
  title        = {Pith review of: Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QREYG6YZ}},
  note         = {Machine review of arXiv:2602.02401}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Human motion analysis tasks, such as temporal 3D pose estimation, motion prediction, and motion in-betweening, play an essential role in computer vision. However, current paradigms suffer from severe fragmentation. First, the field is split between ``perception'' models that understand motion from video but only output text, and ``generation'' models that cannot perceive from raw visual input. Second, generative MLLMs are often limited to single-frame, static poses using dense, parametric SMPL models, failing to handle temporal motion. Third, existing motion vocabularies are built from skeleton data alone, severing the link to the visual domain. To address these challenges, we introduce Superman, a unified framework that bridges visual perception with temporal, skeleton-based motion generation. Our solution is twofold. First, to overcome the modality disconnect, we propose a Vision-Guided Motion Tokenizer. Leveraging the natural geometric alignment between 3D skeletons and visual data, this module pioneers robust joint learning from both modalities, creating a unified, cross-modal motion vocabulary. Second, grounded in this motion language, a single, unified MLLM architecture is trained to handle all tasks. This module flexibly processes diverse, temporal inputs, unifying 3D skeleton pose estimation from video (perception) with skeleton-based motion prediction and in-betweening (generation). Extensive experiments on standard benchmarks, including Human3.6M, demonstrate that our unified method achieves state-of-the-art or competitive performance across all motion tasks. This showcases a more efficient and scalable path for generative motion analysis using skeletons.

Figures

Figures reproduced from arXiv: 2602.02401 by Jason Li, Mengyuan Liu, Peiming Li, Songtao Wu, Xinshun Wang, Zhichao Deng, Zhongbin Fang, Ziyi Wang.

Figure 1
Figure 1. Figure 1: A Unified Framework for Human Motion Perception and Generation. Our unified model achieves state-of-the-art perfor￾mance across three traditionally disparate tasks. Our method consistently outperforms existing methods across all tasks simultaneously: (1) 3D Pose Estimation, (2) Motion Prediction, and (3) Motion In-betweening. Abstract Human motion analysis tasks, such as temporal 3D pose estimation, motion… view at source ↗
Figure 2
Figure 2. Figure 2: Architecture of our Vision-Guided Motion Tokenizer (VGMT). VGMT creates a discrete motion vocabulary by jointly fusing information from two modalities. A Skeleton Encoder (Es) captures geometry while a Visual-Skeleton Attention (VSA) module and a subsequent Visual Encoder (Ev) ground the pose in visual features. The fused representation is quantized against a learnable hybrid codebook, and a decoder recons… view at source ↗
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Two standard pipelines for 3D pose estimation. “2D PE” [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative results for generalizing to motion predic [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Quantitative analysis of the VQ-VAE codebook. (a) Up [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Scaling analysis of model and codebook parameters. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 8 linked inside Pith

  1. [1]

    Posetrack: a benchmark for human pose estimation and tracking

    Mykhaylo Andriluka, Umar Iqbal, Eldar Insafutdinov, Leonid Pishchulin, Anton Milan, Juergen Gall, and Bernt Schiele. Posetrack: a benchmark for human pose estimation and tracking. InCVPR, 2018. 1

    2018

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 5

    Pith/arXiv arXiv 2025

  3. [3]

    Keep it smpl: automatic estimation of 3d human pose and shape from a single image

    Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J Black. Keep it smpl: automatic estimation of 3d human pose and shape from a single image. InECCV, 2016. 7

    2016

  4. [4]

    Motionllm: Understanding human behaviors from human motions and videos.arXiv preprint arXiv:2405.20340, 2024

    Ling-Hao Chen, Shunlin Lu, Ailing Zeng, Hao Zhang, Benyou Wang, Ruimao Zhang, and Lei Zhang. Motionllm: Understanding human behaviors from human motions and videos.arXiv preprint arXiv:2405.20340, 2024. 2, 3

    Pith/arXiv arXiv 2024

  5. [5]

    Cascaded pyramid net- work for multi-person pose estimation

    Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun. Cascaded pyramid net- work for multi-person pose estimation. InProceedings of the IEEE conference on computer vision and pattern recog- nition, pages 7103–7112, 2018. 2, 3

    2018

  6. [6]

    Towards accurate 3d hu- man motion prediction from incomplete observations

    Qiongjie Cui and Huaijiang Sun. Towards accurate 3d hu- man motion prediction from incomplete observations. In CVPR, 2021. 2

    2021

  7. [7]

    Instructblip: Towards general-purpose vision- language models with instruction tuning.Advances in neural information processing systems, 36:49250–49267, 2023

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision- language models with instruction tuning.Advances in neural information processing systems, 36:49250–49267, 2023. 3

    2023

  8. [8]

    Explore in-context learning for 3d point cloud understanding.NeurIPS, 2023

    Zhongbin Fang, Xiangtai Li, Xia Li, Joachim M Buhmann, Chen Change Loy, and Mengyuan Liu. Explore in-context learning for 3d point cloud understanding.NeurIPS, 2023. 2

    2023

  9. [9]

    Posellava: Pose centric multimodal llm for fine-grained 3d pose manipulation

    Dong Feng, Ping Guo, Encheng Peng, Mingmin Zhu, Wen- hao Yu, and Peng Wang. Posellava: Pose centric multimodal llm for fine-grained 3d pose manipulation. InProceedings of the AAAI Conference on Artificial Intelligence, pages 2951– 2959, 2025. 2, 5, 6

    2025

  10. [10]

    Chatpose: Chatting about 3d human pose

    Yao Feng, Jing Lin, Sai Kumar Dwivedi, Yu Sun, Priyanka Patel, and Michael J Black. Chatpose: Chatting about 3d human pose. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2093–2103,

    2093

  11. [11]

    Robust motion in-betweening.ACM TOG,

    F ´elix G Harvey, Mike Yurick, Derek Nowrouzezahrai, and Christopher Pal. Robust motion in-betweening.ACM TOG,

  12. [12]

    Human motion prediction via spatio-temporal in- painting

    Alejandro Hernandez, Jurgen Gall, and Francesc Moreno- Noguer. Human motion prediction via spatio-temporal in- painting. InICCV, 2019. 2

    2019

  13. [13]

    Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: large scale datasets and predic- tive methods for 3d human sensing in natural environments. IEEE T-PAMI, 2013. 2, 5, 6, 7

    2013

  14. [14]

    Motiongpt: Human motion as a foreign lan- guage.Advances in Neural Information Processing Systems, 36:20067–20079, 2023

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as a foreign lan- guage.Advances in Neural Information Processing Systems, 36:20067–20079, 2023. 2, 3, 4, 5, 6, 9

    2023

  15. [15]

    Convolutional autoen- coders for human motion infilling

    Manuel Kaufmann, Emre Aksan, Jie Song, Fabrizio Pece, Remo Ziegler, and Otmar Hilliges. Convolutional autoen- coders for human motion infilling. In3DV, 2020. 2

    2020

  16. [16]

    Unipose: A unified multimodal framework for human pose comprehension, generation and editing

    Yiheng Li, Ruibing Hou, Hong Chang, Shiguang Shan, and Xilin Chen. Unipose: A unified multimodal framework for human pose comprehension, generation and editing. InPro- ceedings of the Computer Vision and Pattern Recognition Conference, pages 27805–27815, 2025. 2, 3, 5, 6

    2025

  17. [17]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023. 3

    2023

  18. [18]

    Recognizing human ac- tions as the evolution of pose estimation maps

    Mengyuan Liu and Junsong Yuan. Recognizing human ac- tions as the evolution of pose estimation maps. InCVPR,

  19. [19]

    Human-in-context: Unified cross-domain 3d human motion modeling via in-context learning.arXiv preprint arXiv:2508.10897, 2025

    Mengyuan Liu, Xinshun Wang, Zhongbin Fang, Deheng Ye, Xia Li, Tao Tang, Songtao Wu, Xiangtai Li, and Ming-Hsuan Yang. Human-in-context: Unified cross-domain 3d human motion modeling via in-context learning.arXiv preprint arXiv:2508.10897, 2025. 2, 5, 6, 7, 8, 3

    Pith/arXiv arXiv 2025

  20. [20]

    Lit- tle

    Julieta Martinez, Rayat Hossain, Javier Romero, and J. Lit- tle. A simple yet effective baseline for 3d human pose esti- mation.2017 IEEE International Conference on Computer Vision (ICCV), pages 2659–2668, 2017. 3

    2017

  21. [21]

    Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli. 3d human pose estimation in video with temporal convolutions and semi-supervised training.2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7745–7754, 2018. 3

    2019

  22. [22]

    Ski models: Skeleton induced vision- language embeddings for understanding activities of daily living

    Arkaprava Sinha, Dominick Reilly, Francois Bremond, Pu Wang, and Srijan Das. Ski models: Skeleton induced vision- language embeddings for understanding activities of daily living. InProceedings of the AAAI Conference on Artificial Intelligence, pages 6931–6939, 2025. 2

    2025

  23. [23]

    Deep high-resolution representation learning for human pose es- timation

    Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose es- timation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5693–5703,

  24. [24]

    In- tegral human pose regression

    Xiao Sun, Bin Xiao, Shuang Liang, and Yichen Wei. In- tegral human pose regression. InEuropean Conference on Computer Vision, 2017. 3

    2017

  25. [25]

    Deeppose: Human pose estimation via deep neural networks.2014 IEEE Con- ference on Computer Vision and Pattern Recognition, pages 1653–1660, 2013

    Alexander Toshev and Christian Szegedy. Deeppose: Human pose estimation via deep neural networks.2014 IEEE Con- ference on Computer Vision and Pattern Recognition, pages 1653–1660, 2013. 3

    2014

  26. [26]

    Neural discrete representation learning.Advances in neural information pro- cessing systems, 30, 2017

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.Advances in neural information pro- cessing systems, 30, 2017. 2

    2017

  27. [27]

    Recovering ac- curate 3d human pose in the wild using imus and a moving camera

    Timo V on Marcard, Roberto Henschel, Michael J Black, Bodo Rosenhahn, and Gerard Pons-Moll. Recovering ac- curate 3d human pose in the wild using imus and a moving camera. InECCV, 2018. 2, 6, 7

    2018

  28. [28]

    Locllm: Exploiting generalizable human keypoint localization via large language model

    Dongkai Wang, Shiyu Xuan, and Shiliang Zhang. Locllm: Exploiting generalizable human keypoint localization via large language model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 5, 6, 3

    2024

  29. [29]

    Gcnext: towards the unity of graph convolutions for human motion prediction

    Xinshun Wang, Qiongjie Cui, Chen Chen, and Mengyuan Liu. Gcnext: towards the unity of graph convolutions for human motion prediction. InAAAI, 2024. 2

    2024

  30. [30]

    Skeleton-in-context: unified skeleton sequence modeling with in-context learning

    Xinshun Wang, Zhongbin Fang, Xia Li, Xiangtai Li, Chen Chen, and Mengyuan Liu. Skeleton-in-context: unified skeleton sequence modeling with in-context learning. In CVPR, 2024. 2, 5, 6, 8

    2024

  31. [31]

    Dynamic dense graph convolutional network for skeleton-based human motion prediction.IEEE T-IP,

    Xinshun Wang, Wanying Zhang, Can Wang, Yuan Gao, and Mengyuan Liu. Dynamic dense graph convolutional network for skeleton-based human motion prediction.IEEE T-IP,

  32. [32]

    mplug-owl: Modularization empowers large language models with multimodality.arXiv preprint arXiv:2304.14178, 2023

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality.arXiv preprint arXiv:2304.14178, 2023. 3

    Pith/arXiv arXiv 2023

  33. [33]

    Llava-pose: Enhancing human pose and action un- derstanding via keypoint-integrated instruction tuning.arXiv preprint arXiv:2506.21317, 2025

    Dewen Zhang, Tahir Hussain, Wangpeng An, and Hayaru Shouno. Llava-pose: Enhancing human pose and action un- derstanding via keypoint-integrated instruction tuning.arXiv preprint arXiv:2506.21317, 2025. 2, 3

    Pith/arXiv arXiv 2025

  34. [34]

    Distribution-aware coordinate representation for human pose estimation

    Feng Zhang, Xiatian Zhu, Hanbin Dai, Mao Ye, and Ce Zhu. Distribution-aware coordinate representation for human pose estimation. InCVPR, 2020. 1, 2

    2020

  35. [35]

    Pose magic: efficient and temporally consis- tent human pose estimation with a hybrid mamba-gcn net- work.arXiv preprint arXiv:2408.02922, 2024

    Xinyi Zhang, Qiqi Bao, Qinpeng Cui, Wenming Yang, and Qingmin Liao. Pose magic: efficient and temporally consis- tent human pose estimation with a hybrid mamba-gcn net- work.arXiv preprint arXiv:2408.02922, 2024. 1, 2, 3

    Pith/arXiv arXiv 2024

  36. [36]

    3d human pose estima- tion with spatial and temporal transformers.2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11636–11645, 2021

    Ce Zheng, Sijie Zhu, Mat’ias Mendieta, Taojiannan Yang, Chen Chen, and Zhengming Ding. 3d human pose estima- tion with spatial and temporal transformers.2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11636–11645, 2021. 3

    2021

  37. [37]

    Generative tweening: long-term inbetween- ing of 3d human motions.arXiv preprint arXiv:2005.08891,

    Yi Zhou, Jingwan Lu, Connelly Barnes, Jimei Yang, Sitao Xiang, et al. Generative tweening: long-term inbetween- ing of 3d human motions.arXiv preprint arXiv:2005.08891,

    Pith/arXiv arXiv 2005

  38. [38]

    Mo- tiongpt3: Human motion as a second modality.arXiv preprint arXiv:2506.24086, 2025

    Bingfan Zhu, Biao Jiang, Sunyi Wang, Shixiang Tang, Tao Chen, Linjie Luo, Youyi Zheng, and Xin Chen. Mo- tiongpt3: Human motion as a second modality.arXiv preprint arXiv:2506.24086, 2025. 2, 5, 6

    arXiv 2025

  39. [39]

    Motionbert: a unified perspective on learning human motion representations

    Wentao Zhu, Xiaoxuan Ma, Zhaoyang Liu, Libin Liu, Wayne Wu, and Yizhou Wang. Motionbert: a unified perspective on learning human motion representations. InICCV, 2023. 2, 5, 6, 7

    2023

  40. [40]

    looks back

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection.arXiv preprint arXiv:2010.04159, 2020. 5 Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation Supplementary Material ... T Visual-Skeleton Attention Vision Encoder Vision Projector ...

    Pith/arXiv arXiv 2010

  41. [41]

    Semantic Spheres

    learn to focus on complementary features—some attend- ing to the heel and others to the toe trajectory—aggregating a robust visual representation despite the noisy input. This confirms that VSA functions not just as a sampler, but as a dynamicvisual corrector. Fig. S6 demonstrates the tracking of the left foot. Even when the 2D estimation is relatively st...