Pith. sign in

REVIEW 4 major objections 5 minor 75 references

Stochastic Human Motion Prediction with Memory of Action Transition and Action Characteristic

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that adding two memory banks—one storing action-transition features, one storing action-characteristic features—to a variational motion-prediction baseline improves action accuracy and distribution fidelity on four…

desk verdict A genuine WAT extension with two new memory banks and an adaptive fusion rule, but the headline gains aren't cleanly attributable because the new ARM cross-entropy loss is never ablated; a WAT+CE baseline is needed before the SOTA claim holds. read the letter →

arxiv 2507.04062 v1 pith:ULXJJX5X submitted 2025-07-05 cs.CV cs.AI

Jianwei Tang , Hong Yang , Tengyue Chen , Jian-Fang Hu This is my paper
classification cs.CVcs.AI
keywords stochastichumanmotionpredictionaction-conditionedgenerationmemorybankssoftretrievalactiontransitionsconditionalvariationalautoencoderdiversityrecognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets action-driven stochastic human motion prediction, where a model sees past frames of one action and must generate several plausible futures of a specified different action. The authors claim that two learned memory banks, STAB and ACB, fix the two failure modes of the WAT baseline: uneven transition speed between actions, and confusion between visually similar actions. They report that their method, with the Adaptive Attention Adjustment fusion strategy, achieves the best action accuracy and FID among compared methods on GRAB, NTU RGB-D, BABEL, and HumanAct12. The stated cost is moderate: frame rate drops from 2.76 to 1.98 FPS and GPU memory rises from 2262 to 2837 MB.

What carries the argument

The central machinery is a pair of key-value memory banks plus an adaptive fusion rule. STAB (Soft-transition Action Bank) stores, for each past-action/future-action pair, $M$ tuples of key and value features representing transition motion; retrieval first indexes by labels and then selects the key most similar to the encoder query, weighting the retrieved value by that similarity, and the soft search repeats this over the top-k recognised past actions and averages by classifier confidence. ACB (Action Characteristic Bank) stores $N$ key-value tuples per future action label and retrieves the most similar characteristic feature. AAA (Adaptive Attention Adjustment) fuses the two retrieved features as $F = \frac{\alpha}{1+\alpha}F_{st} + \frac{1}{1+\alpha}F_{ac}$, where $\alpha$ starts at 1, stays 1 before a time threshold $\tau$, then is updated by the running mean of the cross-entropy loss of the frozen action classifier on generated frames. This time-dependent $\alpha$ is what lets early prediction use transition knowledge and later prediction use action identity.

What would settle it

Replace the ARM's top-k predicted past-action labels with the ground-truth past action at inference, or evaluate generated motions with an independent action classifier not used in training. If the reported Acc and FID gains shrink or vanish, the central claim depends on classifier alignment rather than on the banks' transition and characteristic features.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that action transitions and action characteristics can be externalised into key-value memory banks and retrieved at generation time, and that the mix between transition features and characteristic features should shift as prediction unfolds. The STAB bank is queried by the past and future action labels; because the observed motion may belong to several similar categories, the query uses the top-k recognised past actions and weights each branch by its classifier confidence. The ACB bank is queried by the future action label alone and supplies the distinctive motion pattern of that action. The AAA strategy computes a per-step weight $\alpha$ from the cross-entropy loss of the action classifier on the generated frame, after a time threshold, so early frames lean on transition features and later frames lean on action characteristics. Adding these modules to WAT, the paper reports improved accuracy and FID on all four datasets, with ablations attributing the gain to each component.

Load-bearing premise

The whole pipeline assumes the Action Recognition Module is good enough that its top-k labels point STAB to the relevant transition features, and that using that same classifier's cross-entropy loss on generated frames is a healthy training signal rather than a shortcut.

Editorial extensions

If this is right

  • Adding STAB, ACB, and AAA to the WAT baseline consistently outperforms it on GRAB, NTU, BABEL, and HumanAct12, with the largest stated gains on accuracy and FID.
  • The choice of k in the soft search matters: top-2 gives the best accuracy in the ablation, top-1 loses accuracy because similar actions are not covered, and larger k adds diversity but harms prediction quality.
  • The ablations attribute different roles to the two banks: removing STAB hurts accuracy and diversity, while removing ACB hurts accuracy with little effect on diversity.
  • The method carries a claimed computational cost: frame rate falls from 2.76 to 1.98 FPS and GPU memory use rises from 2262 to 2837 MB; the authors suggest parallel bank retrieval as a remedy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to replace the frozen ARM's top-k labels with ground-truth past action labels at inference; if accuracy and FID gains disappear, the method is compensating for classifier errors rather than learning transitions.
  • The soft top-k retrieval recipe is not specific to motion: any conditional generator whose conditioning label is ambiguous could store transition and identity features in banks and weight retrieved branches by classifier confidence.
  • Because the same ARM is used both as a training loss and as the evaluator for Acc and FID, an independent action classifier or a perceptual study would be needed to confirm that the gains reflect motion quality rather than alignment with that particular recognizer.
  • The time-dependent fusion rule suggests a simple diagnostic: plot $\alpha$ over generation steps; if the learned schedule does not shift from transition features early to characteristic features late, the AAA mechanism's explanation can be tested directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes two memory banks, STAB and ACB, plus an adaptive attention adjustment strategy (AAA), on top of the WAT baseline for action-driven stochastic human motion prediction. STAB stores action-transition features and is queried with a soft top-k search over the predicted past-action distribution; ACB stores per-action characteristic features; AAA adaptively weights the two retrieved features over prediction time. The authors report state-of-the-art Acc and FID on GRAB, NTU, BABEL, and HumanAct12, and ablate components on GRAB. Code and demo are promised. The central claim is that the proposed modules consistently outperform previous SOTA, including the WAT baseline.

Significance. If the central claim held, the paper would make a modest but useful contribution to action-conditioned stochastic motion prediction, a niche but active area. The design is transparent, the experiments span four datasets, and the authors disclose runtime and memory overhead. The availability of code and the use of established evaluation metrics are strengths. However, the significance is bounded by an attribution gap: the proposed method adds a new classifier-based cross-entropy loss at the same time as the memory banks, and the ablations do not separate these changes. The adaptive mechanism in Eq. (8) also lacks derivation and reported hyperparameters, which weakens reproducibility. The diversity metrics do not consistently improve, so the 'consistently outperforms' phrasing is stronger than the evidence.

major comments (4)
  1. [Section 4.3; Table 2] The headline claim that STAB, ACB, and AAA are responsible for the SOTA gains is not cleanly supported because the model also adds a new ARM cross-entropy loss. Section 4.3 states that during MPM training, 'we add the CE loss to the original loss function in [1]' and that generated sequences are classified by the frozen ARM. The ablations in Table 2 remove only STAB, ACB, AAA, or the running-mean update, while the ARM CE loss remains in all rows. Consequently, the w/o-STAB/ACB/AAA rows are not WAT-equivalent; they measure component importance conditional on the CE loss. To validate the contribution, the authors should report a WAT+CE baseline (i.e., the full training pipeline with the ARM CE loss but without STAB, ACB, and AAA) and compare it against both WAT and the full method. Without this, the improvement over WAT cannot be attributed to the proposed banks or AAA.
  2. [Eq. (8); Section 4.3] The adaptive attention mechanism is not fully specified. Eq. (8) defines alpha through an exponential running mean of (CELoss - alpha), but CELoss appears to be treated as a scalar even though cross-entropy is computed per frame, and the update rule as written is not derived from any stated objective. More importantly, the hyperparameters gamma and tau are never reported in the paper, and the default value of k for the soft search in the main experiments is only implicitly indicated by the ablation in Table 3. The bank sizes M and N are also unspecified. Since the method's empirical claims depend on these choices, the manuscript should provide these values in Section 4.3 or in a supplementary table, and should justify or at least discuss the form of Eq. (8).
  3. [Table 1; Section 4.4] The claim that the method 'consistently outperforms' on four datasets is overstated for the diversity metrics. On NTU, Divw is 1.24 vs 1.25 for WAT and Div is 2.19 vs 2.20; on BABEL, Divw ties at 1.35 and Div is 1.72 vs 1.74. These are small but consistently non-improvements. The text in Section 4.4 acknowledges the Div gap on NTU as 'minor' and within error, but the abstract and the concluding sentence of Section 4.4 ('best performance on all four datasets') are stronger than the data support. The central claim is still defensible if the focus is Acc and FID, but the wording should be qualified.
  4. [Section 3.2; Section 4.2; Section 4.3] The dependence of both the STAB retrieval and the MPM training loss on the same frozen ARM raises a potential metric-inflation risk. The Acc reported in Table 1 is computed with a pre-trained recognition model from WAT, while the CE loss during MPM training is computed from a different, newly trained ARM. It is plausible that the CE loss makes generated frames more separable under the ARM's specific decision boundary, which may not transfer to the WAT classifier used for evaluation. This is not a circularity error, but it is a correctness risk. The authors should report Acc with the held-out WAT classifier and preferably also with a third classifier, or at least show that the CE loss alone (the WAT+CE baseline suggested above) does not reproduce the Acc gains.
minor comments (5)
  1. [Table 1] The column header 'Fidte' should be 'FIDte' for consistency with 'FIDtr'; also the caption says 'Results of Act2Mot, DLow, ACTOR and W AT are from [1]' but WAT is sometimes written 'W AT' in the text; please unify the notation.
  2. [Eq. (7) and Eq. (8)] The initial value of alpha is not defined; Eq. (7) uses alpha but Eq. (8) only defines its update after time tau. Please state the initial condition (presumably alpha=1 before tau) explicitly.
  3. [Section 4.1] The NTU RGB-D description says 'a subset containing 13 actions of, with noisy SMPL parameters estimated by VIBE'; the phrase 'of,' appears to be a typo and should be removed.
  4. [Section 3.3.2] In Eq. (5), the superscript '(j)' on Sim and MaxS is defined but not used consistently; please ensure the notation indicates that MaxS^(j) is the maximum over the i index for each j.
  5. [Section 4.6.2] The default value of k is not stated in the main experiments; Table 3 shows top-2 is best on GRAB, but the paper should explicitly say that k=2 is used for all datasets, or report k per dataset if it varies.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an external benchmark comparison and the proposed modules are defined independently of the evaluation metrics.

full rationale

The paper's central claim is an empirical comparison against external baselines (WAT, Act2Mot, DLow, ACTOR) on four motion prediction datasets, so the headline result is not derived from its own assumptions by construction. The proposed STAB and ACB are defined as key-value memory banks queried by similarities with encoder features (Eqs. 4-6), and the AAA fusion weight is a running-mean function of a cross-entropy loss (Eqs. 7-8); none of these equations defines the predicted motion in terms of the evaluation metric or vice versa. The authors do cite two of their own works ([20], [22]) in the related-work section, but these citations are contextual and not load-bearing for the main contribution. The ARM cross-entropy loss introduced in Section 4.3 is an additional unablated training signal, which creates a legitimate attribution gap when claiming that STAB/ACB/AAA alone cause the gains in Table 1; however, this is an experimental confounding issue rather than a circular derivation, because the reported Acc and FID are computed with WAT's pretrained action recognition model, not with the authors' own ARM, and no fitted parameter is renamed as a prediction. On the circularity standard of exhibiting an equation or definition that reduces to its own inputs, no such step is present in the paper.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. The memory banks are model components, not postulated world objects. The load-bearing extras are the ARM reliability assumption and the unspecified AAA hyperparameters.

free parameters (4)
  • top-k value k for soft search = 2 (selected on GRAB)
    Table 3 tests k=1..4 and reports k=2 as the final choice; larger k increases diversity but hurts accuracy.
  • alpha update coefficient gamma = not specified
    Eq. 8 uses gamma in the running-mean update of alpha, but the paper never states its value.
  • time threshold tau = not specified
    Eq. 8 only changes alpha when the predicted time step t exceeds tau; the value of tau is not reported.
  • bank sizes M and N = not specified
    STAB stores M tuples per entry and ACB stores N tuples per entry; neither M nor N is reported.
assumptions (3)
  • domain assumption The preprocessed datasets and evaluation protocol from WAT are valid and unbiased.
    All results use data preprocessed by Mao et al. [1] and the pretrained action recognition model of [1] for Acc and FID (Sections 4.1 and 4.2).
  • domain assumption The ARM's classification of past and generated motions is accurate enough for bank retrieval and training.
    STAB indexes by ARM top-k outputs (Eq. 5), and the MPM is trained with cross-entropy loss from the same ARM (Section 4.3).
  • standard math The CVAE ELBO and reparameterization trick are standard and correctly applied.
    The MPM uses the CVAE framework of [1]; no new derivation is offered beyond Eqs. 1-3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stochastic Human Motion Prediction with Memory of Action Transition and Action Characteristic." pith.science (2026). https://pith.science/paper/ULXJJX5X

@misc{pith2026250704062,
  author       = {Pith},
  title        = {Pith review of: Stochastic Human Motion Prediction with Memory of Action Transition and Action Characteristic},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ULXJJX5X}},
  note         = {Machine review of arXiv:2507.04062}
}
read the original abstract

Action-driven stochastic human motion prediction aims to generate future motion sequences of a pre-defined target action based on given past observed sequences performing non-target actions. This task primarily presents two challenges. Firstly, generating smooth transition motions is hard due to the varying transition speeds of different actions. Secondly, the action characteristic is difficult to be learned because of the similarity of some actions. These issues cause the predicted results to be unreasonable and inconsistent. As a result, we propose two memory banks, the Soft-transition Action Bank (STAB) and Action Characteristic Bank (ACB), to tackle the problems above. The STAB stores the action transition information. It is equipped with the novel soft searching approach, which encourages the model to focus on multiple possible action categories of observed motions. The ACB records action characteristic, which produces more prior information for predicting certain actions. To fuse the features retrieved from the two banks better, we further propose the Adaptive Attention Adjustment (AAA) strategy. Extensive experiments on four motion prediction datasets demonstrate that our approach consistently outperforms the previous state-of-the-art. The demo and code are available at https://hyqlat.github.io/STABACB.github.io/.

Figures

Figures reproduced from arXiv: 2507.04062 by the authors.

Figure 1
Figure 1. Limitations of WAT [1] in predicting action-driven [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our model. The “Enc” and “MLP” represent encoders and multi-layer perceptrons that are the same as [1]. ARM is [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. STAB and ACB structure. In (a), aˆp and af represent the past action label and future action label. Using these two labels, key-parts are retrieved from the bank, and similarity is computed between encoder-derived query features Q and the key-parts. The key￾part with the highest similarity is selected, and its corresponding value-part feature is retrieved. A similarity-driven incentive strategy encourages the model … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: A visualization comparison between our model, WAT, and ground truth on the GRAB dataset. This example illustrates the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: This visualization compares action transitions generated by WAT and our model, focusing on the sequence from “Inspect” [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

75 extracted references · 65 canonical work pages

  1. [1]

    Weakly-supervised ac- tion transition learning for stochastic human motion predic- tion,

    W. Mao, M. Liu, and M. Salzmann, “Weakly-supervised ac- tion transition learning for stochastic human motion predic- tion,” in Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2022, pp. 8151–8160

    work page 2022

  2. [2]

    Anticipating human activities for reactive robotic response

    H. S. Koppula and A. Saxena, “Anticipating human activities for reactive robotic response.” in IROS, vol. 2071. Tokyo, 2013

    work page 2013

  3. [3]

    Neural state machine for character-scene interactions,

    S. Starke, H. Zhang, T. Komura, and J. Saito, “Neural state machine for character-scene interactions,” ACM Transac- tions on Graphics, vol. 38, no. 6, p. 178, 2019

    work page 2019

  4. [4]

    Dlow: Diversifying latent flows for diverse human motion prediction,

    Y . Yuan and K. Kitani, “Dlow: Diversifying latent flows for diverse human motion prediction,” in Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part IX 16. Springer, 2020, pp. 346–364

    work page 2020

  5. [5]

    Real time animation of virtual humans: a trade-off between naturalness and control,

    H. Van Welbergen, B. J. Van Basten, A. Egges, Z. M. Rut- tkay, and M. H. Overmars, “Real time animation of virtual humans: a trade-off between naturalness and control,” in Computer Graphics Forum, vol. 29, no. 8. Wiley Online Library, 2010, pp. 2530–2554

    work page 2010

  6. [6]

    Motion planning,

    S. M. La Valle, “Motion planning,” IEEE Robotics & Au- tomation Magazine, vol. 18, no. 2, pp. 108–118, 2011

    work page 2011

  7. [7]

    Recur- rent network models for human dynamics,

    K. Fragkiadaki, S. Levine, P. Felsen, and J. Malik, “Recur- rent network models for human dynamics,” inProceedings of the IEEE international conference on computer vision, 2015, pp. 4346–4354

    work page 2015

  8. [8]

    A neural temporal model for human motion pre- diction,

    A. Gopalakrishnan, A. Mali, D. Kifer, L. Giles, and A. G. Ororbia, “A neural temporal model for human motion pre- diction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 12 116– 12 125

    work page 2019

Show all 75 references
  1. [9]

    Ad- versarial geometry-aware human motion prediction,

    L.-Y . Gui, Y .-X. Wang, X. Liang, and J. M. Moura, “Ad- versarial geometry-aware human motion prediction,” inPro- ceedings of the european conference on computer vision (ECCV), 2018, pp. 786–803

    2018

  2. [10]

    Structural-rnn: Deep learning on spatio-temporal graphs,

    A. Jain, A. R. Zamir, S. Savarese, and A. Saxena, “Structural-rnn: Deep learning on spatio-temporal graphs,” in Proceedings of the ieee conference on computer vision and pattern recognition, 2016, pp. 5308–5317

    2016

  3. [11]

    On human mo- tion prediction using recurrent neural networks,

    J. Martinez, M. J. Black, and J. Romero, “On human mo- tion prediction using recurrent neural networks,” inProceed- ings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2891–2900

    2017

  4. [12]

    Quaternet: A quaternion-based recurrent model for human motion,

    D. Pavllo, D. Grangier, and M. Auli, “Quaternet: A quaternion-based recurrent model for human motion,” arXiv preprint arXiv:1805.06485, 2018

    2018 arXiv

  5. [13]

    Imitation learning for human pose prediction,

    B. Wang, E. Adeli, H.-k. Chiu, D.-A. Huang, and J. C. Niebles, “Imitation learning for human pose prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7124–7133

    2019

  6. [14]

    Structured pre- diction helps 3d human motion modelling,

    E. Aksan, M. Kaufmann, and O. Hilliges, “Structured pre- diction helps 3d human motion modelling,” in Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, 2019, pp. 7144–7153

    2019

  7. [15]

    Deep representation learning for human motion prediction and classification,

    J. Butepage, M. J. Black, D. Kragic, and H. Kjellstrom, “Deep representation learning for human motion prediction and classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 6158– 6166

    2017

  8. [16]

    Convolutional sequence to sequence model for human dynamics,

    C. Li, Z. Zhang, W. S. Lee, and G. H. Lee, “Convolutional sequence to sequence model for human dynamics,” in Pro- ceedings of the IEEE conference on computer vision and pat- tern recognition, 2018, pp. 5226–5234

    2018

  9. [17]

    History repeats itself: Human motion prediction via motion attention,

    W. Mao, M. Liu, and M. Salzmann, “History repeats itself: Human motion prediction via motion attention,” in Com- puter Vision–ECCV 2020: 16th European Conference, Glas- gow, UK, August 23–28, 2020, Proceedings, Part XIV 16 . Springer, 2020, pp. 474–489

    2020

  10. [18]

    Learning tra- jectory dependencies for human motion prediction,

    W. Mao, M. Liu, M. Salzmann, and H. Li, “Learning tra- jectory dependencies for human motion prediction,” in Pro- ceedings of the IEEE/CVF international conference on com- puter vision, 2019, pp. 9489–9497

    2019

  11. [19]

    Jointly learning heterogeneous features for RGB-D activity recognition,

    J. Hu, W. Zheng, J. Lai, and J. Zhang, “Jointly learning heterogeneous features for RGB-D activity recognition,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 39, no. 11, pp. 2186–2200, 2017. [Online]. Available: https://doi.org/ 10.1109/TPAMI.2016.2640292

    2017

  12. [20]

    Predicting human poses via recurrent attention network,

    J. Tang, J. Wang, and J.-F. Hu, “Predicting human poses via recurrent attention network,” Visual Intelligence, vol. 1, no. 1, p. 18, Aug 2023. [Online]. Available: https://doi.org/10.1007/s44267-023-00020-z

    2023 doi

  13. [21]

    Learning pro- gressive joint propagation for human motion prediction,

    Y . Cai, L. Huang, Y . Wang, T.-J. Cham, J. Cai, J. Yuan, J. Liu, X. Yang, Y . Zhu, X. Shen et al. , “Learning pro- gressive joint propagation for human motion prediction,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VII

    2020

  14. [22]

    Springer, 2020, pp. 226–242

    2020

  15. [23]

    Tem- poral continual learning with prior compensation for human motion prediction,

    J. Tang, J. Sun, X. Lin, W.-S. Zheng, J.-F. Hu et al., “Tem- poral continual learning with prior compensation for human motion prediction,” Advances in Neural Information Pro- cessing Systems, vol. 36, pp. 65 837–65 849, 2023

    2023

  16. [24]

    Motionmixer: Mlp-based 3d human body pose forecasting,

    A. Bouazizi, A. Holzbock, U. Kressel, K. Dietmayer, and V . Belagiannis, “Motionmixer: Mlp-based 3d human body pose forecasting,” inProceedings of the Thirty-First Interna- tional Joint Conference on Artificial Intelligence. Interna- tional Joint Conferences on Artificial Inte...

    2022

  17. [25]

    Long-term hu- man motion prediction by modeling motion context and en- hancing motion dynamic,

    Y . Tang, L. Ma, W. Liu, and W. Zheng, “Long-term hu- man motion prediction by modeling motion context and en- hancing motion dynamic,”arXiv preprint arXiv:1805.02513, 2018

    2018 arXiv

  18. [26]

    Context-aware human motion prediction,

    E. Corona, A. Pumarola, G. Alenya, and F. Moreno-Noguer, “Context-aware human motion prediction,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 6992–7001

    2020

  19. [27]

    Human mo- tion prediction via spatio-temporal inpainting,

    A. Hernandez, J. Gall, and F. Moreno-Noguer, “Human mo- tion prediction via spatio-temporal inpainting,” in Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, 2019, pp. 7134–7143

    2019

  20. [28]

    Space- time-separable graph convolutional network for pose fore- casting,

    T. Sofianos, A. Sampieri, L. Franco, and F. Galasso, “Space- time-separable graph convolutional network for pose fore- casting,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 11 209–11 218

    2021

  21. [29]

    Spatio- temporal gating-adjacency gcn for human motion predic- tion,

    C. Zhong, L. Hu, Z. Zhang, Y . Ye, and S. Xia, “Spatio- temporal gating-adjacency gcn for human motion predic- tion,” in Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2022, pp. 6447–6456

    2022

  22. [30]

    Auto-encoding variational bayes,

    D. P. Kingma, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  23. [31]

    Generative adversarial networks,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020

    2020

  24. [32]

    Motron: Multimodal probabilistic human motion forecasting,

    T. Salzmann, M. Pavone, and M. Ryll, “Motron: Multimodal probabilistic human motion forecasting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 6457–6466

    2022

  25. [33]

    Posegpt: Quantization-based 3d human motion generation and forecasting,

    T. Lucas, F. Baradel, P. Weinzaepfel, and G. Rogez, “Posegpt: Quantization-based 3d human motion generation and forecasting,” in European Conference on Computer Vi- sion. Springer, 2022, pp. 417–435

    2022

  26. [34]

    Behavior-driven synthesis of human dynamics,

    A. Blattmann, T. Milbich, M. Dorkenwald, and B. Ommer, “Behavior-driven synthesis of human dynamics,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 12 236–12 246

    2021

  27. [35]

    Diverse hu- man motion prediction via gumbel-softmax sampling from an auxiliary space,

    L. Dang, Y . Nie, C. Long, Q. Zhang, and G. Li, “Diverse hu- man motion prediction via gumbel-softmax sampling from an auxiliary space,” in Proceedings of the 30th ACM Inter- national Conference on Multimedia, 2022, pp. 5162–5171

    2022

  28. [36]

    Diverse human motion prediction guided by multi-level spatial-temporal anchors,

    S. Xu, Y .-X. Wang, and L.-Y . Gui, “Diverse human motion prediction guided by multi-level spatial-temporal anchors,” in European Conference on Computer Vision . Springer, 2022, pp. 251–269

    2022

  29. [37]

    A stochastic conditioning scheme for diverse hu- man motion prediction,

    S. Aliakbarian, F. S. Saleh, M. Salzmann, L. Petersson, and S. Gould, “A stochastic conditioning scheme for diverse hu- man motion prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 5223–5232

    2020

  30. [38]

    Social gan: Socially acceptable trajectories with generative adversarial networks,

    A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi, “Social gan: Socially acceptable trajectories with generative adversarial networks,” in Proceedings of the IEEE confer- ence on computer vision and pattern recognition , 2018, pp. 2255–2264

    2018

  31. [39]

    Desire: Distant future prediction in dy- namic scenes with interacting agents,

    N. Lee, W. Choi, P. Vernaza, C. B. Choy, P. H. Torr, and M. Chandraker, “Desire: Distant future prediction in dy- namic scenes with interacting agents,” in Proceedings of the IEEE conference on computer vision and pattern recogni- tion, 2017, pp. 336–345

    2017

  32. [40]

    Humanmac: Masked motion completion for human motion prediction,

    L.-H. Chen, J. Zhang, Y . Li, Y . Pang, X. Xia, and T. Liu, “Humanmac: Masked motion completion for human motion prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 9544–9555

    2023

  33. [41]

    Representing cyclic human motion using functional analy- sis,

    D. Ormoneit, M. J. Black, T. Hastie, and H. Kjellstr ¨om, “Representing cyclic human motion using functional analy- sis,” Image and Vision Computing, vol. 23, no. 14, pp. 1264– 1276, 2005

    2005

  34. [42]

    Modeling hu- man locomotion with topologically constrained latent vari- able models,

    R. Urtasun, D. J. Fleet, and N. D. Lawrence, “Modeling hu- man locomotion with topologically constrained latent vari- able models,” in Workshop on Human Motion . Springer, 2007, pp. 104–118

    2007

  35. [43]

    Language2pose: Natural lan- guage grounded pose forecasting,

    C. Ahuja and L.-P. Morency, “Language2pose: Natural lan- guage grounded pose forecasting,” in 2019 International Conference on 3D Vision (3DV). IEEE, 2019, pp. 719–728

    2019

  36. [44]

    Action2motion: Conditioned generation of 3d human motions,

    C. Guo, X. Zuo, S. Wang, S. Zou, Q. Sun, A. Deng, M. Gong, and L. Cheng, “Action2motion: Conditioned generation of 3d human motions,” in Proceedings of the 28th ACM Inter- national Conference on Multimedia, 2020, pp. 2021–2029

    2020

  37. [45]

    Dancing to music,

    H.-Y . Lee, X. Yang, M.-Y . Liu, T.-C. Wang, Y .-D. Lu, M.-H. Yang, and J. Kautz, “Dancing to music,”Advances in neural information processing systems, vol. 32, 2019

    2019

  38. [46]

    Learning to generate diverse dance motions with transformer,

    J. Li, Y . Yin, H. Chu, Y . Zhou, T. Wang, S. Fidler, and H. Li, “Learning to generate diverse dance motions with transformer,” arXiv preprint arXiv:2008.08171, 2020

    2008 arXiv

  39. [47]

    Learn to dance with aist++: Music conditioned 3d dance generation,

    R. Li, S. Yang, D. A. Ross, and A. Kanazawa, “Learn to dance with aist++: Music conditioned 3d dance generation,” arXiv preprint arXiv:2101.08779, vol. 2, no. 3, 2021

    2021 arXiv

  40. [48]

    Human motion modeling using dvgans,

    X. Lin and M. R. Amer, “Human motion modeling using dvgans,” arXiv preprint arXiv:1804.10652, 2018

    2018 arXiv

  41. [49]

    Action-conditioned 3d human motion synthesis with transformer vae,

    M. Petrovich, M. J. Black, and G. Varol, “Action-conditioned 3d human motion synthesis with transformer vae,” in Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10 985–10 995

    2021

  42. [50]

    Audio to body dynamics,

    E. Shlizerman, L. Dery, H. Schoen, and I. Kemelmacher- Shlizerman, “Audio to body dynamics,” in Proceedings of the IEEE conference on computer vision and pattern recog- nition, 2018, pp. 7574–7583

    2018

  43. [51]

    Accurate and diverse sampling of sequences based on a “best of many

    A. Bhattacharyya, B. Schiele, and M. Fritz, “Accurate and diverse sampling of sequences based on a “best of many” sample objective,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8485– 8493

    2018

  44. [52]

    Mt-vae: Learning motion transformations to generate multimodal human dy- namics,

    X. Yan, A. Rastogi, R. Villegas, K. Sunkavalli, E. Shecht- man, S. Hadap, E. Yumer, and H. Lee, “Mt-vae: Learning motion transformations to generate multimodal human dy- namics,” inProceedings of the European conference on com- puter vision (ECCV), 2018, pp. 265–281

    2018

  45. [53]

    Executing your commands via motion diffusion in latent space,

    X. Chen, B. Jiang, W. Liu, Z. Huang, B. Fu, T. Chen, and G. Yu, “Executing your commands via motion diffusion in latent space,” in Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2023, pp. 18 000–18 010

    2023

  46. [54]

    Mofusion: A framework for denoising-diffusion-based mo- tion synthesis,

    R. Dabral, M. H. Mughal, V . Golyanik, and C. Theobalt, “Mofusion: A framework for denoising-diffusion-based mo- tion synthesis,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 9760– 9770

    2023

  47. [55]

    Motiondiffuse: Text-driven human mo- tion generation with diffusion model,

    M. Zhang, Z. Cai, L. Pan, F. Hong, X. Guo, L. Yang, and Z. Liu, “Motiondiffuse: Text-driven human mo- tion generation with diffusion model,” arXiv preprint arXiv:2208.15001, 2022

    2022 arXiv

  48. [56]

    Fg- t2m: Fine-grained text-driven human motion generation via diffusion model,

    Y . Wang, Z. Leng, F. W. Li, S.-C. Wu, and X. Liang, “Fg- t2m: Fine-grained text-driven human motion generation via diffusion model,” in Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision , 2023, pp. 22 035– 22 044

    2023

  49. [57]

    Remodiffuse: Retrieval-augmented motion dif- fusion model,

    M. Zhang, X. Guo, L. Pan, Z. Cai, F. Hong, H. Li, L. Yang, and Z. Liu, “Remodiffuse: Retrieval-augmented motion dif- fusion model,” in Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, 2023, pp. 364–373

    2023

  50. [58]

    Listen, denoise, action! audio-driven motion synthesis with diffusion models,

    S. Alexanderson, R. Nagy, J. Beskow, and G. E. Henter, “Listen, denoise, action! audio-driven motion synthesis with diffusion models,” ACM Transactions on Graphics (TOG) , vol. 42, no. 4, pp. 1–20, 2023

    2023

  51. [59]

    Review on synergizing the metaverse and ai-driven synthetic data: enhancing virtual realms and activity recog- nition in computer vision,

    M. Rajendran, C. T. Tan, I. Atmosukarto, A. B. Ng, and S. See, “Review on synergizing the metaverse and ai-driven synthetic data: enhancing virtual realms and activity recog- nition in computer vision,” Visual Intelligence, vol. 2, no. 1, p. 27, 2024

    2024

  52. [60]

    Diffusion motion: Generate text-guided 3d human motion by diffusion model,

    Z. Ren, Z. Pan, X. Zhou, and L. Kang, “Diffusion motion: Generate text-guided 3d human motion by diffusion model,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  53. [61]

    Understanding textdriven motion synthesis with keyframe collaboration via diffusion models,

    D. Wei, X. Sun, H. Sun, B. Li, S. Hu, W. Li, and J. Lu, “Understanding textdriven motion synthesis with keyframe collaboration via diffusion models,”arXiv preprint arXiv:2305.13773, vol. 3, 2023

    2023 arXiv

  54. [62]

    Mod- iff: Action-conditioned 3d motion generation with de- noising diffusion probabilistic models,

    M. Zhao, M. Liu, B. Ren, S. Dai, and N. Sebe, “Mod- iff: Action-conditioned 3d motion generation with de- noising diffusion probabilistic models,” arXiv preprint arXiv:2301.03949, 2023

    2023 arXiv

  55. [63]

    Cg-hoi: Contact-guided 3d human-object interaction generation,

    C. Diller and A. Dai, “Cg-hoi: Contact-guided 3d human-object interaction generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 19 888–19 901

    2024

  56. [64]

    Pose guided human video generation,

    C. Yang, Z. Wang, X. Zhu, C. Huang, J. Shi, and D. Lin, “Pose guided human video generation,” inProceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 201–216

    2018

  57. [65]

    Smpl: A skinned multi-person linear model,

    M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “Smpl: A skinned multi-person linear model,” in Seminal Graphics Papers: Pushing the Boundaries, Volume 2, 2023, pp. 851–866

    2023

  58. [66]

    Con- tactdb: Analyzing and predicting grasp contact via thermal imaging,

    S. Brahmbhatt, C. Ham, C. C. Kemp, and J. Hays, “Con- tactdb: Analyzing and predicting grasp contact via thermal imaging,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 8709– 8719

    2019

  59. [67]

    Grab: A dataset of whole-body human grasping of objects,

    O. Taheri, N. Ghorbani, M. J. Black, and D. Tzionas, “Grab: A dataset of whole-body human grasping of objects,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16. Springer, 2020, pp. 581–600

    2020

  60. [68]

    Ntu rgb+ d 120: A large-scale benchmark for 3d human activity understanding,

    J. Liu, A. Shahroudy, M. Perez, G. Wang, L.-Y . Duan, and A. C. Kot, “Ntu rgb+ d 120: A large-scale benchmark for 3d human activity understanding,”IEEE transactions on pattern analysis and machine intelligence, vol. 42, no. 10, pp. 2684– 2701, 2019

    2019

  61. [69]

    Ntu rgb+ d: A large scale dataset for 3d human activity analysis,

    A. Shahroudy, J. Liu, T.-T. Ng, and G. Wang, “Ntu rgb+ d: A large scale dataset for 3d human activity analysis,” inPro- ceedings of the IEEE conference on computer vision and pat- tern recognition, 2016, pp. 1010–1019

    2016

  62. [70]

    Babel: Bodies, ac- tion and behavior with english labels,

    A. R. Punnakkal, A. Chandrasekaran, N. Athanasiou, A. Quiros-Ramirez, and M. J. Black, “Babel: Bodies, ac- tion and behavior with english labels,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 722–731

    2021

  63. [71]

    Vibe: Video inference for human body pose and shape estimation,

    M. Kocabas, N. Athanasiou, and M. J. Black, “Vibe: Video inference for human body pose and shape estimation,” in Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, 2020, pp. 5253–5263

    2020

  64. [72]

    Amass: Archive of motion capture as surface shapes,

    N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. J. Black, “Amass: Archive of motion capture as surface shapes,” in Proceedings of the IEEE/CVF international con- ference on computer vision, 2019, pp. 5442–5451

    2019

  65. [73]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems, vol. 30, 2017

    2017

  66. [74]

    Predict- ing 3d human dynamics from video,

    J. Y . Zhang, P. Felsen, A. Kanazawa, and J. Malik, “Predict- ing 3d human dynamics from video,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 7114–7123

    2019

  67. [75]

    Adam: A method for stochastic optimiza- tion,

    P. K. Diederik, “Adam: A method for stochastic optimiza- tion,” (No Title), 2014

    2014

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.