Pith. sign in

REVIEW 3 major objections 4 minor 4 cited by

Lifting Motion to the 3D World via 2D Diffusion

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a four-stage 2D diffusion pipeline estimates global 3D motion from a single-view 2D pose sequence using only 2D pose training data, and beats prior work, including 3D-supervised methods, on five datasets.

desk verdict MVLift is a real advance in 2D-only lifting, but the 'no 3D supervision' claim is overstated because VPoser/SMAL carry 3D priors. read the letter →

arxiv 2411.18808 v2 pith:AZJGCFSY submitted 2024-11-27 cs.CV

Jiaman Li , C. Karen Liu , Jiajun Wu This is my paper · ORCID
classification cs.CV
keywords 3Dmotionestimation2Dposeliftingdiffusionmodelmulti-viewconsistencyepipolargeometryroottrajectoryanimalhuman-objectinteraction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles a bottleneck in 3D motion estimation: most methods need paired 3D ground truth or mocap-scale datasets, which do not exist for activities like gymnastics or animal play. MVLift is a four-stage framework that uses only 2D pose sequences for training and still predicts global 3D motion, meaning joint rotations and root trajectories in a world coordinate frame. The central claim is that multi-view consistency can be learned progressively through 2D motion diffusion, first with epipolar line conditioning, then joint optimization with score distillation, then synthetic strictly consistent multi-view data, and finally a single-pass multi-view generator. If correct, 3D motion capture becomes unnecessary for building motion estimators on new domains, and the same recipe works for humans, animals, and human-object interactions.

What carries the argument

The load-bearing mechanism is a progression of 2D motion diffusion models that construct multi-view consistency without ever seeing a synchronized multi-view video. Stage 1 trains a diffusion model conditioned on epipolar lines, with each joint constrained to lie on a line derived from a virtual epipole during training or from essential matrices between camera views at test time. Stage 2 optimizes five unobserved views with a multi-view consistency loss plus Score Distillation Sampling from that diffusion prior. Stage 3 recovers 3D joints by reprojection, fits SMPL or SMAL parameters with VPoser, and reprojects to make a synthetic dataset of strictly consistent multi-view sequences. Stage 4 trains a final diffusion model that generates all views at once using cross-view attention, so a single forward pass yields the multi-view sequences needed for 3D reconstruction. This progression converts a single-view 2D input into geometrically consistent pseudo-observations, turning 3D lifting into a triangulation-and-reprojection problem solvable without 3D labels.

What would settle it

Run MVLift on a held-out sequence with known 3D ground truth while scaling the assumed camera intrinsics K by a factor of two; if the predicted root trajectory does not scale accordingly, the metric interpretation is not actually recovered. Alternatively, test on an animal with body proportions far outside the SMAL training distribution, such as a giraffe, and check whether the SMPL or SMAL fitting stage can represent the motion; failure would show the method inherits the parametric model's coverage limits.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a 2D-only training signal is sufficient to lift a single-view 2D pose sequence into a global 3D motion, provided the model can synthesize geometrically consistent multi-view 2D sequences. MVLift does this by training a line-conditioned 2D motion diffusion model on epipolar constraints, optimizing multi-view sequences with Score Distillation Sampling and a pairwise epipolar consistency loss, fitting SMPL or SMAL parameters to the resulting 3D joints via VPoser, reprojecting to create strictly consistent synthetic multi-view data, and then training a multi-view 2D diffusion model with cross-view attention. The result, the paper reports, is lower root-translation error, lower joint position error, and better 2D reprojection quality than existing methods on AIST++, Steezy, NicoleMove, CatPlay, and OMOMO, including 3D-supervised baselines such as MotionBERT and WHAM.

Load-bearing premise

The load-bearing premise is that the pretrained parametric body models and pose priors, SMPL, SMAL, and VPoser, are accurate enough when fitted to the recovered 3D joints to supply plausible joint rotations and a meaningful metric scale, despite those models themselves having been trained on 3D data.

Editorial extensions

If this is right

  • On domains where 3D motion capture is unavailable or expensive, such as dance, yoga, gymnastics, or animal behavior, a motion estimator can now be trained from monocular 2D pose data alone.
  • The predicted output includes root trajectories in world coordinates, so applications like character animation, robotics, and sports analysis get global motion, not just joint angles.
  • The same multi-stage recipe transfers across humans, animals, and human-object interactions without changing the training objective, only the parametric body model used for fitting.
  • The comparison results imply that the 2D diffusion prior captures enough 3D structure that explicit paired 3D supervision is not the limiting factor for these datasets.
  • A single forward pass at inference, after training the final multi-view generator, makes the approach practical for processing large 2D pose datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the camera intrinsics are specified rather than estimated, the metric scale of the recovered root trajectories is tied to the assumed K; a natural extension is to estimate K from the video or predict it jointly with motion.
  • Editorial inference: the method's transfer to animals is bounded by the coverage of SMPL and SMAL; species with body proportions far outside those parametric models would likely need a category-specific model, and the 2D-only claim would then be conditional on that model's availability.
  • Editorial inference: the Stage-3 synthetic dataset makes the final generator's quality depend on the realism of Stage-2 optimization, so one could test the pipeline by ablating Stage 2's SDS weight and measuring how the final end-to-end error changes.
  • Editorial inference: the epipolar setup assumes a known relative camera arrangement, such as 60-degree or 90-degree intervals, so applying the method to footage with unknown or moving cameras would require relaxing this assumption, possibly by predicting relative poses.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces MVLift, a four-stage pipeline that takes a single-view 2D pose sequence as input and predicts global 3D motion, including joint rotations and root trajectories in a world coordinate system. In Stage 1, a diffusion model is trained to generate 2D pose sequences conditioned on simulated epipolar lines. Stage 2 jointly optimizes multi-view 2D sequences using Score Distillation Sampling and a multi-view consistency loss. Stage 3 fits SMPL/SMAL parameters with VPoser to the recovered 3D joints, then reprojects the fitted 3D motions into strictly consistent multi-view 2D sequences to form a synthetic dataset. Stage 4 trains a multi-view diffusion model on this synthetic data to directly generate consistent multi-view 2D sequences. The method is evaluated on human dance (AIST++, Steezy, NicoleMove), animal (CatPlay), and human-object interaction (OMOMO) datasets, with ablations on AIST++. The core claim is that MVLift outperforms prior work, including methods trained with 3D supervision, without itself using 3D motion data or paired video-3D data for training.

Significance. If the core claim holds, MVLift would be a meaningful step toward 3D motion estimation in domains where 3D ground truth is scarce, such as animal motion and complex sports. The multi-stage design is coherent, and the ablations in Table 3 show that each stage contributes to the final result. The external evaluation against 3D ground truth on AIST++ breaks the self-referential loop of the synthetic training data, which is a strength. However, the no-3D-supervision claim is materially weakened by the use of pre-trained 3D body models and priors (SMPL, SMAL, VPoser) in Stage 3. The significance of the contribution is therefore real but narrower than the abstract suggests: the paper demonstrates that 3D-trained priors can be combined with 2D-only training of the diffusion stages to yield strong global 3D motion, not that no 3D-trained component is involved.

major comments (3)
  1. [Sec. 2, Sec. 3.3] The central claim that the method 'does not rely on any 3D motion data or paired video and 3D data for training' and uses 'only 2D pose sequences' is contradicted by Stage 3, which explicitly fits SMPL parameters using VPoser [35] and, for animals, uses SMAL [67]. VPoser is a variational prior trained on AMASS, and SMAL is trained on 3D Menagerie; both encode metric bone lengths, plausible joint rotations, and pose-dependent shape information. Because a single monocular 2D sequence does not determine metric scale, the metric root trajectory reported in Table 1 is made possible by these 3D-trained priors. The correct claim is that no paired video-to-3D supervisison is used for the diffusion stages, not that no 3D-trained model is used. I recommend reframing the headline claim and, if possible, adding an ablation that removes or replaces VPoser/SMPL to quantify how much of the metric root trajectory is inherited from these priors.
  2. [Sec. 3.1, Sec. 3.2, Table 1] The paper specifies a virtual camera rig with a chosen intrinsic matrix K and essential matrices E_v encoding relative rotations and translations, but the input 2D pose sequence is extracted from a real monocular video with an unknown camera. The manuscript does not explain how the real camera is aligned to the virtual coordinate system, nor how the metric scale of the recovered root trajectory is fixed and compared to ground truth in Table 1. Since Troot is reported in metric units, the evaluation must include the alignment procedure (for example, whether the predicted trajectory is globally scaled or aligned to the ground truth before computing the error). Without this description, the reported Troot advantage over WHAM and MotionBERT is not interpretable, and the reader cannot tell whether the metric scale is learned from 2D multi-view consistency or simply inherited from the specified virtual baseline and the SMPL/SMAL skeleton.
  3. [Sec. 4.5, Table 3] The ablations effectively show the contribution of the line-conditioned diffusion and the SDS loss, but they do not isolate the contribution of the 3D-trained body models. In particular, the 'SDS for 3D, w/o lepi' row demonstrates that epipolar conditioning is crucial, but there is no ablation that removes VPoser/SMPL from Stage 3 (for example, by directly triangulating the optimized multi-view 2D sequences and smoothing the resulting 3D joints). Because the no-3D-supervision claim is the paper's main selling point, this missing ablation is load-bearing: it leaves open the possibility that the metric accuracy of the root trajectory is almost entirely supplied by VPoser and the parametric body models, rather than by the multi-view consistency machinery introduced in this paper.
minor comments (4)
  1. [Sec. 4.1] The definitions of J2D and JC2D do not specify which camera views are included in the error computation. If the input view is included, the error can be trivially small for any method that predicts the input 2D poses; the paper should state explicitly whether J2D is averaged over input and novel views or only over novel views.
  2. [Eq. (5) and Eq. (6)] The normalization factor 1/(2M) in Eq. (5) is inconsistent with the definition of L_{line}^{(v->w)} in Eq. (6), which does not contain a factor of 1/2. Please clarify whether the line distance is meant to be symmetric or whether the factor 2 accounts for the two directions used in the sum.
  3. [Sec. 4.2, Figure 5] The perceptual study plot is hard to read because the raw preference counts are shown without confidence intervals or a statistical test. Reporting percentages with error bars or a paired test would make the preference claim more robust.
  4. [References] References [35] and [36] are the same paper (Expressive Body Capture) duplicated; please consolidate.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central multi-view-consistency derivation is self-contained and externally validated; only minor self-referential data-generation and non-load-bearing self-citations are present.

full rationale

MVLift's derivation is not circular under the seven defined patterns. The pipeline contains an internal self-referential loop: Stage 2's optimized multi-view 2D sequences are lifted to 3D in Stage 3, reprojected into strictly consistent views, and then used to train the Stage 4 multi-view diffusion model. However, this is a data-generation loop rather than an equivalence-by-construction between inputs and predictions: the real single-view 2D pose sequence remains the source evidence, the epipolar and multi-view consistency constraints are geometric rather than fitted to the test quantity, and the final predictions are evaluated against external 3D ground truth on AIST++ and OMOMO, which breaks any statistical tautology. The pretrained SMPL/SMAL and VPoser models are external priors trained on 3D data; their use weakens the literal 'no 3D supervision' claim in Section 2, but that is a scope or correctness concern, not a circular reduction of a prediction to its input. The paper's self-citations, including CIRCLE, the Steezy dance-transformer paper, OMOMO, and EDGE, are dataset or architecture references and are not load-bearing for the central multi-view-consistency derivation. No specific equation or fitted parameter was found to be equivalent by construction to the claimed output, so no circular step is quoted.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three kinds of load-bearing assumptions: hand-chosen camera parameters that define the virtual rig, pretrained 3D-body priors that supply metric scale and pose plausibility, and the generalization ability of 2D diffusion models to out-of-distribution multi-view generation. The virtual epipole is a mathematical construction, not a new physical entity, so no invented entities are counted.

free parameters (3)
  • Camera intrinsic matrix K = unspecified
    Chosen by hand for all virtual views; it determines the epipolar line geometry and, together with the essential matrices, the scale and orientation of the reconstructed 3D motion.
  • Virtual camera arrangement = 5 views at 60-degree intervals in Stage 2, 4 views at 90-degree intervals in Stage 3
    The chosen camera baselines define the multi-view geometry, but no justification or sensitivity analysis is provided.
  • SDS and multi-view consistency loss weights = not reported
    The weights balancing motion realism and geometric consistency in Stage 2 are chosen by hand and not reported, leaving the optimization under-specified.
assumptions (4)
  • ad hoc to paper Epipolar geometry with a fixed, known virtual camera rig is a valid model of multi-view consistency for the input 2D pose sequence.
    The input comes from an arbitrary real camera, but the method treats it as view 0 of a synthetic circular rig with specified intrinsics and essential matrices. This rig is introduced for this paper.
  • domain assumption Pretrained parametric body models (SMPL, SMAL) and VPoser provide a sufficient 3D prior to resolve metric scale and plausible joint rotations.
    The no-3D-supervision claim depends on these models, which were trained on 3D ground truth data. Section 3.3 explicitly uses VPoser for SMPL fitting.
  • domain assumption A 2D motion diffusion model trained on in-domain single-view 2D poses can generate realistic out-of-distribution multi-view 2D sequences under epipolar line conditions.
    Stages 1 and 4 rely on this generalization; no proof is given, and it is the core empirical bet of the method.
  • domain assumption The 2D pose detectors (ViTPose, UniPose) provide accurate, noise-free 2D keypoints.
    The pipeline is trained on extracted 2D poses, so detector errors propagate; the paper does not analyze robustness to noisy keypoints.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lifting Motion to the 3D World via 2D Diffusion." pith.science (2026). https://pith.science/paper/AZJGCFSY

@misc{pith2026241118808,
  author       = {Pith},
  title        = {Pith review of: Lifting Motion to the 3D World via 2D Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AZJGCFSY}},
  note         = {Machine review of arXiv:2411.18808}
}
read the original abstract

Estimating 3D motion from 2D observations is a long-standing research challenge. Prior work typically requires training on datasets containing ground truth 3D motions, limiting their applicability to activities well-represented in existing motion capture data. This dependency particularly hinders generalization to out-of-distribution scenarios or subjects where collecting 3D ground truth is challenging, such as complex athletic movements or animal motion. We introduce MVLift, a novel approach to predict global 3D motion -- including both joint rotations and root trajectories in the world coordinate system -- using only 2D pose sequences for training. Our multi-stage framework leverages 2D motion diffusion models to progressively generate consistent 2D pose sequences across multiple views, a key step in recovering accurate global 3D motion. MVLift generalizes across various domains, including human poses, human-object interactions, and animal poses. Despite not requiring 3D supervision, it outperforms prior work on five datasets, including those methods that require 3D supervision.

Figures

Figures reproduced from arXiv: 2411.18808 by the authors.

Figure 1
Figure 1. Our framework, MVLift, can be trained only on 2D pose sequences and generate 3D motions including joint rotations and root [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our multi-stage framework. In Stage 1, we train a 2D motion diffusion model conditioned on simulated epipolar lines. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Denoising network of the multi-view 2D motion diffusion [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative results of AIST++. On the left, we show the trajectories for each component [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Results of human perceptual studies. den and unrealistic depth changes due to unresolved depth ambiguity. Human Perceptual Study. We conducted a human per￾ceptual study to complement the evaluation of generated 3D motion quality. For each dataset (AIST++, Steezy, Nicol…
Figure 6
Figure 6. Figure 6: Qualitative result comparisons of CatPlay. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative result comparisons of OMOMO. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MoRAE: Flow-Friendly Self-Supervised Latents for Text-to-Motion Generation

    cs.CV 2026-07 conditional novelty 7.0 of 10

    Distilling frozen Motion-JEPA features into a compact 32-D latent whose geometry is coupled to the decoder lets a standard non-autoregressive flow-matching DiT reach state-of-the-art text-to-motion quality on HumanML3...

  2. Absolute Coordinates Make Motion Generation Easy

    cs.CV 2025-05 conditional novelty 7.0 of 10

    Using absolute 3D joint coordinates with a plain Transformer and velocity-prediction diffusion outperforms the standard local-relative motion representation, improving fidelity and enabling direct control.

  3. Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression

    cs.CV 2024-11 conditional novelty 7.0 of 10

    A masked-autoregressive diffusion model trained on a compact essential-feature latent space claims state-of-the-art text-to-motion generation under a new essential-dimension evaluation protocol.

  4. Sketch2Anim: Towards Transferring Sketch Storyboards into 3D Animation

    cs.GR 2025-04 conditional novelty 6.0 of 10

    Sketch2Anim aligns 2D sketch keyposes and joint trajectories with 3D embeddings and uses a trajectory ControlNet plus keypose adapter to generate 3D motion clips from storyboards.

Reference graph

Works this paper leans on

67 extracted references · 54 canonical work pages · cited by 4 Pith papers

  1. [14]

    Mas: Multi-view ancestral sampling for 3d motion generation using 2d diffusion

    Roy Kapon, Guy Tevet, Daniel Cohen-Or, and Amit H Bermano. Mas: Multi-view ancestral sampling for 3d motion generation using 2d diffusion. In CVPR, 2024. 2, 5, 6, 7

    work page 2024

  2. [35]

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3d hands, face, and body from a single image. In CVPR, 2019. 5

    2019

  3. [67]

    Silvia Zuffi, Angjoo Kanazawa, David Jacobs, and Michael J. Black. 3D menagerie: Modeling the 3D shape and pose of animals. In CVPR, 2017. 7

    work page 2017

  4. [1]

    CIRCLE: Capture in rich contextual environ- ments

    Joao Pedro Araujo, Jiaman Li, Karthik Vetrivel, Rishi Agar- wal, Deepak Gopinath, Jiajun Wu, Alexander Clegg, and C Karen Liu. CIRCLE: Capture in rich contextual environ- ments. In CVPR, 2023. 1

    work page 2023

  5. [2]

    Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J. Black. Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image. In ECCV, 2016. 2, 6, 7

    work page 2016

  6. [3]

    Exploit- ing spatial-temporal relationships for 3d pose estimation via graph convolutional networks

    Yujun Cai, Liuhao Ge, Jun Liu, Jianfei Cai, Tat-Jen Cham, Junsong Yuan, and Nadia Magnenat Thalmann. Exploit- ing spatial-temporal relationships for 3d pose estimation via graph convolutional networks. In ICCV, 2019. 2

    work page 2019

  7. [4]

    Generative novel view synthesis with 3d-aware diffusion models

    Eric R Chan, Koki Nagano, Matthew A Chan, Alexander W Bergman, Jeong Joon Park, Axel Levy, Miika Aittala, Shalini De Mello, Tero Karras, and Gordon Wetzstein. Generative novel view synthesis with 3d-aware diffusion models. In ICCV, 2023. 3

    2023

  8. [5]

    Beyond static features for temporally consistent 3d human pose and shape from a video

    Hongsuk Choi, Gyeongsik Moon, Ju Yong Chang, and Ky- oung Mu Lee. Beyond static features for temporally consistent 3d human pose and shape from a video. In CVPR, 2021. 2

    work page 2021

Show all 67 references
  1. [6]

    Optimizing network structure for 3d human pose estimation

    Hai Ci, Chunyu Wang, Xiaoxuan Ma, and Yizhou Wang. Optimizing network structure for 3d human pose estimation. In ICCV, 2019. 2

    2019

  2. [7]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In CVPR, 2023. 3

    2023

  3. [8]

    Humans in 4d: Recon- structing and tracking humans with transformers

    Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran, Angjoo Kanazawa, and Jitendra Malik. Humans in 4d: Recon- structing and tracking humans with transformers. In ICCV,

  4. [9]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020. 4

    2020

  5. [10]

    Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: Large scale datasets and predic- tive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelli- gence, 36(7):1325–1339, 2013. 2, 3

    2013

  6. [11]

    Scaling up dynamic human-scene interaction modeling

    Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, and Siyuan Huang. Scaling up dynamic human-scene interaction modeling. In CVPR, 2024. 1

    2024

  7. [12]

    End-to-end recovery of human shape and pose

    Angjoo Kanazawa, Michael J Black, David W Jacobs, and Jitendra Malik. End-to-end recovery of human shape and pose. In CVPR, 2018. 2

    2018

  8. [13]

    Learning 3d human dynamics from video

    Angjoo Kanazawa, Jason Y Zhang, Panna Felsen, and Jiten- dra Malik. Learning 3d human dynamics from video. In CVPR, 2019. 2

    2019

  9. [15]

    Muhammed Kocabas, Nikos Athanasiou, and Michael J. Black. Vibe: Video inference for human body pose and shape estimation. In CVPR, 2020. 1, 2

    2020

  10. [16]

    Pare: Part attention regressor for 3d human body estimation

    Muhammed Kocabas, Chun-Hao P Huang, Otmar Hilliges, and Michael J Black. Pare: Part attention regressor for 3d human body estimation. In ICCV, 2021. 2

    2021

  11. [17]

    Spec: Seeing people in the wild with an estimated camera

    Muhammed Kocabas, Chun-Hao P Huang, Joachim Tesch, Lea M ¨uller, Otmar Hilliges, and Michael J Black. Spec: Seeing people in the wild with an estimated camera. In ICCV,

  12. [18]

    Pace: Human and motion estimation from in-the-wild videos

    Muhammed Kocabas, Ye Yuan, Pavlo Molchanov, Yunrong Guo, Michael J Black, Otmar Hilliges, Jan Kautz, and Umar Iqbal. Pace: Human and motion estimation from in-the-wild videos. 3DV, 2024. 2

    2024

  13. [19]

    Learning to reconstruct 3d human pose and shape via model-fitting in the loop

    Nikos Kolotouros, Georgios Pavlakos, Michael J Black, and Kostas Daniilidis. Learning to reconstruct 3d human pose and shape via model-fitting in the loop. In ICCV, 2019. 2

    2019

  14. [20]

    Collab- orative video diffusion: Consistent multi-video generation with camera control

    Zhengfei Kuang, Shengqu Cai, Hao He, Yinghao Xu, Hong- sheng Li, Leonidas Guibas, and Gordon Wetzstein. Collab- orative video diffusion: Consistent multi-video generation with camera control. arXiv preprint arXiv:2405.17414, 2024. 3

    2024 arXiv

  15. [21]

    Learning to generate diverse dance motions with transformer

    Jiaman Li, Yihang Yin, Hang Chu, Yi Zhou, Tingwu Wang, Sanja Fidler, and Hao Li. Learning to generate diverse dance motions with transformer. arXiv preprint arXiv:2008.08171,

    2008 arXiv

  16. [22]

    Hybrik: A hybrid analytical-neural inverse kinematics solution for 3d human pose and shape estimation

    Jiefeng Li, Chao Xu, Zhicun Chen, Siyuan Bian, Lixin Yang, and Cewu Lu. Hybrik: A hybrid analytical-neural inverse kinematics solution for 3d human pose and shape estimation. In CVPR, 2021. 2

    2021

  17. [23]

    Niki: Neural inverse kinematics with invertible neural networks for 3d human pose and shape estimation

    Jiefeng Li, Siyuan Bian, Qi Liu, Jiasheng Tang, Fan Wang, and Cewu Lu. Niki: Neural inverse kinematics with invertible neural networks for 3d human pose and shape estimation. In CVPR, 2023. 2

    2023

  18. [24]

    Object motion guided human motion synthesis

    Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis. ACM Trans. Graph., 42(6), 2023. 1, 4, 6, 8

    2023

  19. [25]

    Ai choreographer: Music conditioned 3d dance generation with aist++

    Ruilong Li, Shan Yang, David A Ross, and Angjoo Kanazawa. Ai choreographer: Music conditioned 3d dance generation with aist++. In ICCV, 2021. 5, 6, 8

    2021

  20. [26]

    Mhformer: Multi-hypothesis transformer for 3d human pose estimation

    Wenhao Li, Hong Liu, Hao Tang, Pichao Wang, and Luc Van Gool. Mhformer: Multi-hypothesis transformer for 3d human pose estimation. In CVPR, 2022. 2

    2022

  21. [27]

    Zero-1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. In ICCV, 2023. 3

    2023

  22. [28]

    Syncdreamer: Gener- ating multiview-consistent images from a single-view image

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gener- ating multiview-consistent images from a single-view image. arXiv preprint arXiv:2309.03453, 2023. 3

    2023 arXiv

  23. [29]

    Wonder3d: Single im- age to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Single im- age to 3d using cross-domain diffusion. In CVPR, 2024. 3

    2024

  24. [30]

    Smpl: A skinned multi- person linear model

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multi- person linear model. ACM transactions on graphics (TOG), 34(6):1–16, 2015. 2, 6

    2015

  25. [31]

    3d human motion estimation via motion compression and refine- ment

    Zhengyi Luo, S Alireza Golestaneh, and Kris M Kitani. 3d human motion estimation via motion compression and refine- ment. In ACCV, 2020. 2

    2020

  26. [32]

    AMASS: Archive of motion capture as surface shapes

    Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Ger- ard Pons-Moll, and Michael J Black. AMASS: Archive of motion capture as surface shapes. In ICCV, 2019. 1, 2, 3, 6

    2019

  27. [33]

    A simple yet effective baseline for 3d human pose estimation

    Julieta Martinez, Rayat Hossain, Javier Romero, and James J Little. A simple yet effective baseline for 3d human pose estimation. In ICCV, 2017. 2

    2017

  28. [34]

    Monocular 3d human pose estimation in the wild using im- proved cnn supervision

    Dushyant Mehta, Helge Rhodin, Dan Casas, Pascal Fua, Olek- sandr Sotnychenko, Weipeng Xu, and Christian Theobalt. Monocular 3d human pose estimation in the wild using im- proved cnn supervision. In 3DV, 2017. 2, 3

    2017

  29. [36]

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from a single image. In CVPR, 2019. 2

    2019

  30. [37]

    3d human pose estimation in video with tem- poral convolutions and semi-supervised training

    Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli. 3d human pose estimation in video with tem- poral convolutions and semi-supervised training. In CVPR,

  31. [38]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 5

    2022 arXiv

  32. [39]

    P-stmo: Pre-trained spatial temporal many-to-one model for 3d human pose estimation

    Wenkang Shan, Zhenhua Liu, Xinfeng Zhang, Shanshe Wang, Siwei Ma, and Wen Gao. P-stmo: Pre-trained spatial temporal many-to-one model for 3d human pose estimation. In ECCV,

  33. [40]

    Global-to-local modeling for video-based 3d human pose and shape estimation

    Xiaolong Shen, Zongxin Yang, Xiaohan Wang, Jianxin Ma, Chang Zhou, and Yi Yang. Global-to-local modeling for video-based 3d human pose and shape estimation. In CVPR,

  34. [41]

    Zero123++: a single image to consistent multi-view diffusion base model

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view diffusion base model. arXiv preprint arXiv:2310.15110, 2023. 3

    2023 arXiv

  35. [42]

    Mvdream: Multi-view diffusion for 3d gener- ation

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gener- ation. arXiv preprint arXiv:2308.16512, 2023. 3

    2023 arXiv

  36. [43]

    Wham: Reconstructing world-grounded humans with accu- rate 3d motion

    Soyong Shin, Juyong Kim, Eni Halilaj, and Michael J Black. Wham: Reconstructing world-grounded humans with accu- rate 3d motion. In CVPR, 2024. 1, 2, 3, 6

    2024

  37. [44]

    Hu- maneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion

    Leonid Sigal, Alexandru O Balan, and Michael J Black. Hu- maneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. International journal of computer vision, 87(1):4–27, 2010. 2

    2010

  38. [45]

    Trace: 5d temporal regression of avatars with dynamic cam- eras in 3d environments

    Yu Sun, Qian Bao, Wu Liu, Tao Mei, and Michael J Black. Trace: 5d temporal regression of avatars with dynamic cam- eras in 3d environments. In CVPR, 2023. 2

    2023

  39. [46]

    Black, and Dim- itrios Tzionas

    Omid Taheri, Nima Ghorbani, Michael J. Black, and Dim- itrios Tzionas. GRAB: A dataset of whole-body human grasp- ing of objects. In ECCV, 2020. 1

    2020

  40. [47]

    Make-it-3d: High-fidelity 3d creation from a single image with diffusion prior

    Junshu Tang, Tengfei Wang, Bo Zhang, Ting Zhang, Ran Yi, Lizhuang Ma, and Dong Chen. Make-it-3d: High-fidelity 3d creation from a single image with diffusion prior. In ICCV,

  41. [48]

    Human motion diffusion model

    Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Amit H Bermano, and Daniel Cohen-Or. Human motion diffusion model. In ICLR, 2023. 4

    2023

  42. [49]

    Diffusion with forward models: Solv- ing stochastic inverse problems without direct supervision

    Ayush Tewari, Tianwei Yin, George Cazenavette, Semon Rezchikov, Josh Tenenbaum, Fr´edo Durand, Bill Freeman, and Vincent Sitzmann. Diffusion with forward models: Solv- ing stochastic inverse problems without direct supervision. NeurIPS, 2023. 3

    2023

  43. [50]

    EDGE: Editable dance generation from music

    Jonathan Tseng, Rodrigo Castellon, and C Karen Liu. EDGE: Editable dance generation from music. In CVPR, 2023. 4

    2023

  44. [51]

    Generative camera dolly: Extreme monocular dynamic novel view synthesis

    Basile Van Hoorick, Rundi Wu, Ege Ozguroglu, Kyle Sar- gent, Ruoshi Liu, Pavel Tokmakov, Achal Dave, Changxi Zheng, and Carl V ondrick. Generative camera dolly: Extreme monocular dynamic novel view synthesis. arXiv preprint arXiv:2405.14868, 2024. 3

    2024 arXiv

  45. [52]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, 2017. 2, 4

    2017

  46. [53]

    Encoder-decoder with multi-level attention for 3d human shape and pose estimation

    Ziniu Wan, Zhengjia Li, Maoqing Tian, Jianbo Liu, Shuai Yi, and Hongsheng Li. Encoder-decoder with multi-level attention for 3d human shape and pose estimation. In ICCV,

  47. [54]

    Elepose: Unsupervised 3d human pose estimation by predicting camera elevation and learning normalizing flows on 2d poses

    Bastian Wandt, James J Little, and Helge Rhodin. Elepose: Unsupervised 3d human pose estimation by predicting camera elevation and learning normalizing flows on 2d poses. In CVPR, 2022. 2, 6

    2022

  48. [55]

    Motion guided 3d pose estimation from videos

    Jingbo Wang, Sijie Yan, Yuanjun Xiong, and Dahua Lin. Motion guided 3d pose estimation from videos. In ECCV,

  49. [56]

    Capturing humans in motion: Temporal- attentive 3d human pose and shape estimation from monocular video

    Wen-Li Wei, Jen-Chun Lin, Tyng-Luh Liu, and Hong- Yuan Mark Liao. Capturing humans in motion: Temporal- attentive 3d human pose and shape estimation from monocular video. In CVPR, 2022. 2

    2022

  50. [57]

    Vitpose: Simple vision transformer baselines for human pose estimation

    Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. Vitpose: Simple vision transformer baselines for human pose estimation. NeurIPS, 2022. 5, 6

    2022

  51. [58]

    Dmv3d: Denoising multi-view diffusion using 3d large reconstruction model

    Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Jiahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, et al. Dmv3d: Denoising multi-view diffusion using 3d large reconstruction model. arXiv preprint arXiv:2311.09217,

    arXiv

  52. [59]

    Unipose: Detecting any keypoints

    Jie Yang, Ailing Zeng, Ruimao Zhang, and Lei Zhang. Unipose: Detecting any keypoints. arXiv preprint arXiv:2310.08530, 2023. 6

    2023 arXiv

  53. [60]

    Decoupling human and camera motion from videos in the wild

    Vickie Ye, Georgios Pavlakos, Jitendra Malik, and Angjoo Kanazawa. Decoupling human and camera motion from videos in the wild. In CVPR, 2023. 2, 3

    2023

  54. [61]

    Glamr: Global occlusion-aware human mesh recovery with dynamic cameras

    Ye Yuan, Umar Iqbal, Pavlo Molchanov, Kris Kitani, and Jan Kautz. Glamr: Global occlusion-aware human mesh recovery with dynamic cameras. In CVPR, 2022. 2, 3

    2022

  55. [62]

    Pymaf-x: Towards well-aligned full-body model regression from monocular im- ages

    Hongwen Zhang, Yating Tian, Yuxiang Zhang, Mengcheng Li, Liang An, Zhenan Sun, and Yebin Liu. Pymaf-x: Towards well-aligned full-body model regression from monocular im- ages. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10):12287–12303, 2023. 2

    2023

  56. [63]

    Mixste: Seq2seq mixed spatio-temporal encoder for 3d human pose estimation in video

    Jinlu Zhang, Zhigang Tu, Jianyu Yang, Yujin Chen, and Jun- song Yuan. Mixste: Seq2seq mixed spatio-temporal encoder for 3d human pose estimation in video. In CVPR, 2022. 2

    2022

  57. [64]

    Real-time monocular full- body capture in world space via sequential proxy-to-motion learning

    Yuxiang Zhang, Hongwen Zhang, Liangxiao Hu, Hongwei Yi, Shengping Zhang, and Yebin Liu. Real-time monocular full- body capture in world space via sequential proxy-to-motion learning. arXiv preprint arXiv:2307.01200, 2023. 2

    2023 arXiv

  58. [65]

    3d human pose estimation with spatial and temporal transformers

    Ce Zheng, Sijie Zhu, Matias Mendieta, Taojiannan Yang, Chen Chen, and Zhengming Ding. 3d human pose estimation with spatial and temporal transformers. In ICCV, 2021. 2

    2021

  59. [66]

    Motionbert: A unified perspective on learning human motion representations

    Wentao Zhu, Xiaoxuan Ma, Zhaoyang Liu, Libin Liu, Wayne Wu, and Yizhou Wang. Motionbert: A unified perspective on learning human motion representations. In ICCV, 2023. 1, 2, 3, 6

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.