Pith. sign in

REVIEW 3 major objections 6 minor 78 references

GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusion

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GeoMan produces temporally consistent depth and normal maps from monocular human videos by conditioning a video diffusion model on a first-frame geometry estimate.

desk verdict Clean I2V decomposition and root-relative depth produce genuine temporal-consistency wins, but the unqualified SOTA claim is undercut by the paper's own Goliath normal numbers. read the letter →

arxiv 2505.23085 v1 pith:NSEAMKFR submitted 2025-05-29 cs.CV cs.AIeess.IV

Gwanghyun Kim , Xueting Li , Ye Yuan , Koki Nagano , Tianye Li , Jan Kautz , Se Young Chun , Umar Iqbal This is my paper
classification cs.CVcs.AIeess.IV
keywords humangeometryestimationvideodepthsurfacenormalimage-to-videodiffusionroot-relativetemporalconsistencymonocular4Ddata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that estimating 3D human geometry (depth and surface normals) from monocular video can be reframed as an image-to-video generation problem, and that doing so yields temporally consistent results without large collections of 4D human data. The authors propose a two-stage pipeline: an image-based model (I2G) estimates depth and normals for the first frame, and a video diffusion model (V2G) conditioned on that first-frame estimate produces geometry for the whole sequence. To preserve real human size, they introduce a root-relative depth representation that subtracts the depth of the pelvis root from metric depth, keeping scale while discarding only global translation. If correct, this would make high-quality, flicker-free human geometry estimation practical from ordinary cameras, with metric depth recoverable by adding a pose-estimated root depth.

What carries the argument

The central machinery is the image-to-video generation formulation combined with a human-centered root-relative depth representation. The I2G model (an image latent diffusion model repurposed for depth and normal estimation) provides the first-frame geometry; the V2G model (a pre-trained image-to-video diffusion model, initialized with Ctrl-Adapter's ControlNet, with the denoiser fine-tuned) generates the full video geometry conditioned on the RGB frames and the first-frame geometry. The root-relative depth identity $D_{\text{root-rel}} = D_{\text{metric}} - d_{\text{root}}$ carries the scale-preservation argument: it removes only the global translation, keeps metric scale, and makes the learning target bounded by human height instead of unbounded scene depth. Unified depth and normal training is achieved by switching only the conditioning reference image.

What would settle it

Measure I2G's first-frame normal and depth error against ground truth on a real-world human video benchmark, then correlate it with the full GeoMan video error on the same sequences; if videos with large first-frame errors still show GeoMan matching the GT-conditioned V2G numbers from Table 5(d), the claimed first-frame gating would be falsified.

Watch

Extended reading notes

Core claim

GeoMan's central claim is that video geometry estimation for human subjects is best decomposed into image geometry estimation followed by image-to-video synthesis, rather than learned end-to-end from scarce 4D data. The I2G module estimates depth or normal maps for the first video frame using a latent diffusion model trained on synthetic renders; the V2G module, built on a pre-trained image-to-video diffusion model with a ControlNet-style conditioning, generates geometry for all frames conditioned on the RGB video and the first-frame geometry. A single V2G model handles both depth and normals by switching only the conditioning reference image. For depth, the paper proposes root-relative depth, defined as $D_{\text{root-rel}} = D_{\text{metric}} - d_{\text{root}}$ where $d_{\text{root}}$ is the depth of the pelvis joint; this preserves human scale, restricts the value range to about $[-h/2, h/2]$, and enables metric depth recovery by adding a pose-estimated root depth. The authors report state-of-the-art accuracy and temporal consistency on ActorsHQ and Goliath, outperforming baselines including Sapiens, which was trained on proprietary data.

Load-bearing premise

GeoMan's end-to-end accuracy is gated by the I2G first-frame estimate, which is trained only on synthetic THuman-2.0 renders and is not quantitatively validated on real images.

Editorial extensions

If this is right

  • Monocular human videos can yield metric-scale depth maps, not just affine-invariant ones, by combining root-relative prediction with a pose-estimated pelvis depth.
  • Training video geometry models requires far less 4D data than prior video-depth methods, since the heavy lifting is done by the image model and the pre-trained video prior.
  • A single video diffusion model can serve both depth and normal estimation, improving generalization through joint training.
  • Temporal consistency improves because the root-relative representation avoids per-frame min/max normalization, removing a source of flicker.
  • The gap between I2G+V2G and GT+V2G shown in Table 5(d) implies that improving the first-frame image estimator directly improves whole-video geometry.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The first-frame gating result suggests a modular path: any improvement in image-based human geometry estimation, such as better synthetic-to-real transfer or self-supervised fine-tuning, should transfer almost linearly to video quality, an implicit recommendation the paper does not state.
  • The root-relative depth idea could carry over to other categories with a defined root, such as animals or vehicles, where a semantic keypoint acts as $d_{\text{root}}$; the paper does not explore this.
  • One testable extension is to replace the pose-estimated root depth with a learned root-depth regressor trained on real data; Table 3 shows that using ground-truth root depth improves $\delta_1$ from 0.972 to 0.998, indicating headroom.
  • The reliance on matting and pose estimation listed in the limitations means the end-to-end metric pipeline's real-world behavior is bounded by those upstream components; the video diffusion stabilizes geometry but does not fix a bad segmentation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. GeoMan estimates temporally consistent depth and normal maps for monocular human videos by decomposing the task into an image-to-geometry (I2G) model that predicts first-frame geometry and a video-to-geometry (V2G) image-to-video diffusion model that generates geometry for the full sequence conditioned on that first-frame prediction. The method also introduces a root-relative depth representation that preserves human scale while discarding global translation. Training uses only public data (THuman-2.0, XHumans), and zero-shot evaluations on ActorsHQ and Goliath, plus in-the-wild qualitative results, are reported. The paper claims state-of-the-art accuracy and temporal consistency, and supports the claim with extensive ablations, including comparisons against a naïve video-diffusion extension, multimodal vs. unimodal training, and first-frame conditioning studies.

Significance. If the central engineering claims hold, GeoMan offers a practical way to adapt image-to-video diffusion priors to human geometry estimation with very limited 4D training data, and the root-relative depth representation is a simple, potentially reusable idea for preserving metric human scale. The paper's strengths include a clean task decomposition, thorough ablations, evaluation on multiple datasets, and the use of optical-flow-based temporal consistency metrics. However, the unqualified state-of-the-art accuracy claim is not uniformly supported by the paper's own numbers, and the small margins on the primary benchmark are not accompanied by any uncertainty or significance analysis.

major comments (3)
  1. [Abstract, Sec. 5.1, Table S7] The unqualified claim that GeoMan “achieves state-of-the-art performance in both qualitative and quantitative evaluations” is contradicted by the paper's own supplementary evaluation. In Table S7, on the Goliath dataset, Sapiens outperforms GeoMan on all four surface-normal accuracy metrics (mean angular error 11.205 vs. 12.831, median 8.856 vs. 10.034, 11.25° accuracy 64.284 vs. 56.665, and 30° accuracy 96.091 vs. 94.308), with GeoMan winning only the temporal-consistency metrics. The abstract and conclusion should be revised to claim state-of-the-art temporal consistency with competitive or leading accuracy, and the discussion in Sec. 5.1 should address the Goliath result explicitly.
  2. [Table 2, Sec. 5.1] On the primary ActorsHQ benchmark, GeoMan's accuracy advantage over Sapiens is very small (mean angular error 16.185 vs. 16.278, a relative difference of about 0.6%), and Sapiens actually achieves higher 30° accuracy on both the moving-subject (89.177 vs. 87.784) and moving-camera (89.617 vs. 88.789) splits. No error bars, multiple-seed results, or significance tests are reported anywhere in the paper, despite inference being stochastic (100 diffusion steps with an ensemble of 8). Given the small margins, the authors should report variance over seeds or otherwise demonstrate that the accuracy ordering is not within sampling noise.
  3. [Sec. 5.1, Table 2, Table S7] The paper repeatedly contrasts GeoMan with Sapiens* (Sapiens fine-tuned on the authors' dataset) to argue for superiority under a “fair” setting, while the original Sapiens, which is the actual state-of-the-art baseline, remains competitive or superior depending on dataset and metric. The claim that GeoMan “outperforms state-of-the-art methods despite using only public data” is therefore misleading as stated. The authors should separate the fairness argument from the state-of-the-art claim and clearly report where original Sapiens beats GeoMan (Goliath accuracy) and where GeoMan wins (temporal consistency).
minor comments (6)
  1. [Sec. 4.1] There is a typo in Sec. 4.1: “As as result” should read “As a result”.
  2. [Supplementary A.3, Eq. S5] The threshold set in Eq. S5 is written as t ∈ {11.5°, 20°}, but the main paper consistently uses 11.25° and 30°; the supplementary should be corrected to match the main text.
  3. [Table S8(d)] In the row for “Only 3D data” in Table S8(d), the TC-Mean column contains a stray opening bracket before the value 9.966, which appears to be a formatting error.
  4. [Supplementary B.5] The first-frame dependency discussion says “as shown in Tab. 4 in the paper,” but the relevant ablation is Table 5(d) in the main paper; the cross-reference should be corrected.
  5. [Sec. 4.1, Eq. (2)] Depth maps are defined as D^{(1:F)} ∈ R^{F×3×H×W} in Sec. 4.1, which suggests three channels for depth, while depth is normally single-channel; please clarify whether depth is treated as a 3-channel map or correct the shape notation.
  6. [Figures S11–S16] The captions of Figures S11–S16 state that “GeoMan produces the state-of-the-art results”; these captions should be softened to reflect the quantitative findings, where Sapiens is competitive or better on accuracy for Goliath.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method is trained and evaluated on disjoint public benchmarks, and the SOTA overclaim is a consistency/correctness issue, not a circular-reasoning one.

full rationale

GeoMan is an empirical systems paper rather than a derivation. Its load-bearing components are defined constructively: the I2G model estimates first-frame geometry, the V2G model is trained with the objective in Eq. (2) conditioned on that first-frame estimate, and the root-relative depth representation is explicitly defined as D_root-rel = D_metric - d_root in Sec. 4.2. Training uses public datasets (THuman-2.0 and XHumans), while evaluation uses disjoint benchmarks (ActorsHQ and Goliath), and no model parameter is fitted to the test set. The first-frame ablation in Table 5(d) (GT+V2G vs I2G+V2G) is an honest upper-bound analysis, not a masked fit, and the paper explicitly frames it as showing potential for future gains rather than as the source of its reported accuracy. The self-citations present (e.g., GLAMR as an optional root-depth source) are not load-bearing; the root depth used in evaluation comes from the external TRAM/VIMO method [58]. The paper's abstract-level claim of state-of-the-art performance is internally weakened by its own supplementary Table S7, where Sapiens beats GeoMan on all four normal-accuracy metrics on Goliath, and by the absence of significance tests around the small ActorsHQ lead over Sapiens. That is a correctness and internal-consistency concern, not a circularity concern, because the comparison numbers come from fixed checkpoints and external data rather than from a parameter that was defined in terms of the target result. No step in the claimed chain reduces by construction to its own inputs, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities. The listed items are hand-set constants and transfer assumptions, which are the real load-bearing premises of the empirical method.

free parameters (3)
  • h (max human height) in root-relative depth range = not specified
    Used to clamp root-relative depth to [-h/2, h/2]; chosen by hand as a normalization design choice, not derived from data.
  • Ensemble size for inference = 8
    Number of denoising runs averaged at inference; chosen for quality/speed trade-off and affects reported metrics.
  • Training iterations for I2G/V2G = 20K/30K
    Stopping points selected by validation; longer training gives diminishing or slightly negative returns in the ablations.
assumptions (4)
  • domain assumption Pretrained Stable Diffusion 2 and I2VGen-XL contain transferable priors that remain useful for geometry estimation
    Sec. A.1 initializes I2G and V2G from these checkpoints; if the priors do not transfer, the data-efficiency claim collapses.
  • domain assumption SMPL-X pelvis annotations and VIMO/TRAM pose estimates provide sufficiently accurate root depth
    Sec. 4.2 and Table 3 use root depth from pose estimation to convert root-relative predictions to metric depth; the limitation is acknowledged in Sec. C.
  • domain assumption Synthetic renderings from THuman-2.0 and XHumans generalize to real in-the-wild videos
    Sec. 4.3 trains only on public 3D/4D scans; in-the-wild evaluation is qualitative only, so this generalization is assumed rather than quantitatively demonstrated.
  • domain assumption BiRefNet matting accurately isolates humans in the wild
    Sec. A.1 preprocesses in-the-wild videos with BiRefNet matting; the paper lists matting dependence as a limitation in Sec. C.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusion." pith.science (2026). https://pith.science/paper/NSEAMKFR

@misc{pith2026250523085,
  author       = {Pith},
  title        = {Pith review of: GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NSEAMKFR}},
  note         = {Machine review of arXiv:2505.23085}
}
read the original abstract

Estimating accurate and temporally consistent 3D human geometry from videos is a challenging problem in computer vision. Existing methods, primarily optimized for single images, often suffer from temporal inconsistencies and fail to capture fine-grained dynamic details. To address these limitations, we present GeoMan, a novel architecture designed to produce accurate and temporally consistent depth and normal estimations from monocular human videos. GeoMan addresses two key challenges: the scarcity of high-quality 4D training data and the need for metric depth estimation to accurately model human size. To overcome the first challenge, GeoMan employs an image-based model to estimate depth and normals for the first frame of a video, which then conditions a video diffusion model, reframing video geometry estimation task as an image-to-video generation problem. This design offloads the heavy lifting of geometric estimation to the image model and simplifies the video model's role to focus on intricate details while using priors learned from large-scale video datasets. Consequently, GeoMan improves temporal consistency and generalizability while requiring minimal 4D training data. To address the challenge of accurate human size estimation, we introduce a root-relative depth representation that retains critical human-scale details and is easier to be estimated from monocular inputs, overcoming the limitations of traditional affine-invariant and metric depth representations. GeoMan achieves state-of-the-art performance in both qualitative and quantitative evaluations, demonstrating its effectiveness in overcoming longstanding challenges in 3D human geometry estimation from videos.

Figures

Figures reproduced from arXiv: 2505.23085 by the authors.

Figure 1
Figure 1. GeoMan provides accurate and temporally stable geo [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of GeoMan: (a) Given a video sequence [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of depth representations: Our representa [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Zero-shot normal estimation comparison on ActorsHQ. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Comparison with existing depth estimation models. GeoMan achieves state-of-the-art performance in both depth prediction (top [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Comparison of naive extension and GeoMan across [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

78 extracted references · 56 canonical work pages

  1. [1]

    Photorealistic monocular 3d reconstruction of humans wear- ing clothing

    Thiemo Alldieck, Mihai Zanfir, and Cristian Sminchisescu. Photorealistic monocular 3d reconstruction of humans wear- ing clothing. CVPR, 2022. 2

    work page 2022

  2. [2]

    Lumiere: A space- time diffusion model for video generation

    Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Her- rmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, et al. Lumiere: A space- time diffusion model for video generation. arXiv preprint arXiv:2401.12945, 2024. 3

    arXiv 2024

  3. [3]

    Zoedepth: Zero-shot trans- fer by combining relative and metric depth

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth. arXiv preprint arXiv:2302.12288, 2023. 2, 3, 5, 7

    arXiv 2023

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv:2311.15127, 2023. 2, 3, 4

    arXiv 2023

  5. [5]

    Richter, and Vladlen Koltun

    Aleksei Bochkovskii, Ama ¨el Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R. Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second. arXiv:2410.02073, 2024. 2, 3, 5, 7, 8

    arXiv 2024

  6. [6]

    Video generation models as world simulators,

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators,

  7. [7]

    High accuracy optical flow estimation based on a theory for warping

    Thomas Brox, Andr ´es Bruhn, Nils Papenberg, and Joachim Weickert. High accuracy optical flow estimation based on a theory for warping. ECCV, 2004. 6

    work page 2004

  8. [8]

    Stable- video: Text-driven consistency-aware diffusion video edit- ing

    Wenhao Chai, Xun Guo, Gaoang Wang, and Yan Lu. Stable- video: Text-driven consistency-aware diffusion video edit- ing. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 23040–23050, 2023. 3

    work page 2023

Show all 78 references
  1. [9]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512, 2023

    arXiv

  2. [10]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. CVPR, 2024. 3

    2024

  3. [11]

    Self-supervised learning with geometric constraints in monocular video: Connecting flow, depth, and camera

    Yuhua Chen, Cordelia Schmid, and Cristian Sminchis- escu. Self-supervised learning with geometric constraints in monocular video: Connecting flow, depth, and camera. ICCV, 2019. 2

    2019

  4. [12]

    Cogview2: Faster and better text-to-image generation via hi- erarchical transformers

    Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to-image generation via hi- erarchical transformers. NeurIPS, 2022. 3

    2022

  5. [13]

    Depth map prediction from a single image using a multi-scale deep net- work

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep net- work. NeurIPS, 2014. 12

    2014

  6. [14]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. ICCV, 2023. 3

    2023

  7. [15]

    GeoWiz- ard: Unleashing the diffusion priors for 3d geometry estima- tion from a single image

    Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. GeoWiz- ard: Unleashing the diffusion priors for 3d geometry estima- tion from a single image. ECCV, 2024. 2, 3, 6, 7

    2024

  8. [16]

    Humans in 4D: Re- constructing and tracking humans with transformers

    Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran, Angjoo Kanazawa, and Jitendra Malik. Humans in 4D: Re- constructing and tracking humans with transformers. ICCV,

  9. [17]

    High-fidelity 3d human digitization from single 2k resolution images

    Sang-Hun Han, Min-Gyu Park, Ju Hong Yoon, Ju-Mi Kang, Young-Jae Park, and Hae-Gon Jeon. High-fidelity 3d human digitization from single 2k resolution images. CVPR, 2023. 2, 4

    2023

  10. [18]

    Lotus: Diffusion-based visual foundation model for high-quality dense prediction

    Jing He, Haodong Li, Wei Yin, Yixun Liang, Leheng Li, Kaiqiang Zhou, Hongbo Liu, Bingbing Liu, and Ying- Cong Chen. Lotus: Diffusion-based visual foundation model for high-quality dense prediction. arXiv preprint arXiv:2409.18124, 2024. 1, 3, 6, 7

    2024 arXiv

  11. [19]

    Latent video diffusion models for high-fidelity video generation with arbitrary lengths

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity video generation with arbitrary lengths. In arXiv preprint arXiv:2211.13221, 2022. 3

    2022 arXiv

  12. [20]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. NeurIPS, 2020. 3 9

    2020

  13. [21]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. arXiv:2204.03458, 2022. 3

    2022 arXiv

  14. [22]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation

    Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In arXiv,

  15. [23]

    Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation. arXiv preprint arXiv:2404.15506, 2024. 2, 7

    2024 arXiv

  16. [24]

    DepthCrafter: Generating consistent long depth sequences for open-world videos

    Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xi- aodong Cun, Yong Zhang, Long Quan, and Ying Shan. DepthCrafter: Generating consistent long depth sequences for open-world videos. arXiv preprint arXiv:2409.02095 ,

    arXiv

  17. [25]

    ARCH: Animatable reconstruction of clothed humans

    Zeng Huang, Yuanlu Xu, Christoph Lassner, Hao Li, and Tony Tung. ARCH: Animatable reconstruction of clothed humans. CVPR, 2020. 2

    2020

  18. [26]

    Flowformer: A transformer architecture for optical flow

    Zhaoyang Huang, Xiaoyu Shi, Chao Zhang, Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer: A transformer architecture for optical flow. In European conference on computer vision, pages 668–685. Springer, 2022. 13

    2022

  19. [27]

    HumanRF: High-fidelity neural radiance fields for humans in motion

    Mustafa Is ¸ık, Martin R ¨unz, Markos Georgopoulos, Taras Khakhulin, Jonathan Starck, Lourdes Agapito, and Matthias Nießner. HumanRF: High-fidelity neural radiance fields for humans in motion. ACM Transactions on Graphics (TOG), 42(4):1–12, 2023. 6

    2023

  20. [28]

    Jafarian and H

    Y . Jafarian and H. Park. Self-supervised 3d representa- tion learning of dressed humans from social media videos. TPAMI, 2022. 6

    2022

  21. [29]

    Learning high fidelity depths of dressed humans by watching social media dance videos

    Yasamin Jafarian and Hyun Soo Park. Learning high fidelity depths of dressed humans by watching social media dance videos. CVPR, 2021. 1, 2, 5, 6, 7

    2021

  22. [30]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. CVPR, 2024. 1, 2, 3, 4, 5, 7, 8, 18

    2024

  23. [31]

    Sapiens: Foundation for human vision mod- els

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els. ECCV, 2024. 1, 2, 5, 6, 7, 8, 15

    2024

  24. [32]

    Auto-encoding variational bayes

    DP Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 3, 4

    2013 arXiv

  25. [33]

    Adam: A method for stochastic opti- mization

    Diederik P Kingma. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 12

    2014 arXiv

  26. [34]

    Robust consistent video depth estimation

    Johannes Kopf, Xuejian Rong, and Jia-Bin Huang. Robust consistent video depth estimation. CVPR, 2021. 2

    2021

  27. [35]

    Ctrl- Adapter: An efficient and versatile framework for adapt- ing diverse controls to any diffusion model

    Han Lin, Jaemin Cho, Abhay Zala, and Mohit Bansal. Ctrl- Adapter: An efficient and versatile framework for adapt- ing diverse controls to any diffusion model. arXiv preprint arXiv:2404.09967, 2024. 3, 4, 12

    2024 arXiv

  28. [36]

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi- person linear model. SIGGRAPH Asia, 2015. 2

    2015

  29. [37]

    Consistent video depth estimation.ACM Transactions on Graphics (TOG), 39(4), 2020

    Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. Consistent video depth estimation.ACM Transactions on Graphics (TOG), 39(4), 2020. 2

    2020

  30. [38]

    Jewett, Simon Ven- shtain, Christopher Heilman, Yueh-Tung Chen, Sidi Fu, Mo- hamed Ezzeldin A

    Julieta Martinez, Emily Kim, Javier Romero, Timur Bagaut- dinov, Shunsuke Saito, Shoou-I Yu, Stuart Anderson, Michael Zollh ¨ofer, Te-Li Wang, Shaojie Bai, Chenghui Li, Shih-En Wei, Rohan Joshi, Wyatt Borsos, Tomas Simon, Jason Saragih, Paul Theodosis, Alexander Greene, Anjani...

    2024

  31. [39]

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from a single image. CVPR, 2019. 2, 5

    2019

  32. [40]

    UniDepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniDepth: Universal monocular metric depth estimation. CVPR, 2024. 2, 3, 7, 8

    2024

  33. [41]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. ICLR, 2023. 3

    2023

  34. [42]

    Hierarchical text-conditional image gen- eration with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents. arXiv preprint arXiv:2204.06125,

    arXiv

  35. [43]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. TPAMI, 44(3):1623–1637, 2020. 2

    2020

  36. [44]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. CVPR, 2022. 3

    2022

  37. [45]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. CVPR, 2023. 3

    2023

  38. [46]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS, 2022. 3 10

    2022

  39. [47]

    Pifu: Pixel-aligned implicit function for high-resolution clothed human digitiza- tion

    Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Mor- ishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitiza- tion. CVPR, 2019. 1, 2

    2019

  40. [48]

    Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization

    Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. CVPR, 2020. 1, 2

    2020

  41. [49]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. ICLR, 2022. 3, 5

    2022

  42. [50]

    X-Avatar: Ex- pressive human avatars

    Kaiyue Shen, Chen Guo, Manuel Kaufmann, Juan Zarate, Julien Valentin, Jie Song, and Otmar Hilliges. X-Avatar: Ex- pressive human avatars. CVPR, 2023. 5

    2023

  43. [51]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. ICML, 2015. 3

    2015

  44. [52]

    Diffusers: State-of-the-art diffu- sion models

    Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffu- sion models. https://github.com/huggingface/ diffusers...

    2022

  45. [53]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. In arXiv, 2023. 3

    2023

  46. [54]

    4D-DRESS: A 4d dataset of real-world human clothing with semantic annotations

    Wenbo Wang, Hsuan-I Ho, Chen Guo, Boxiang Rong, Artur Grigorev, Jie Song, Juan Jose Zarate, and Otmar Hilliges. 4D-DRESS: A 4d dataset of real-world human clothing with semantic annotations. CVPR, 2024. 5

    2024

  47. [55]

    MagicVideo-V2: Multi- stage high-aesthetic video generation

    Weimin Wang, Jiawei Liu, Zhijie Lin, Jiangqiao Yan, Shuo Chen, Chetwin Low, Tuyen Hoang, Jie Wu, Jun Hao Liew, Hanshu Yan, et al. MagicVideo-V2: Multi- stage high-aesthetic video generation. arXiv preprint arXiv:2401.04468, 2024. 3

    2024 arXiv

  48. [56]

    Videocomposer: Compositional video synthesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: Compositional video synthesis with motion controllability. NeurIPS, 2023. 3

    2023

  49. [57]

    Less is more: Consistent video depth estimation with masked frames modeling

    Yiran Wang, Zhiyu Pan, Xingyi Li, Zhiguo Cao, Ke Xian, and Jianming Zhang. Less is more: Consistent video depth estimation with masked frames modeling. In ACM MM ,

  50. [58]

    TRAM: Global trajectory and motion of 3d humans from in- the-wild videos

    Yufu Wang, Ziyun Wang, Lingjie Liu, and Kostas Daniilidis. TRAM: Global trajectory and motion of 3d humans from in- the-wild videos. In Proc. European Conf. on Computer Vi- sion (ECCV), pages 467–487, 2024. 2, 5, 8, 12

    2024

  51. [59]

    ICON: Implicit clothed humans obtained from nor- mals

    Yuliang Xiu, Jinlong Yang, Dimitrios Tzionas, and Michael J Black. ICON: Implicit clothed humans obtained from nor- mals. CVPR, 2022. 2

    2022

  52. [60]

    Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, and Michael J. Black. ECON: Explicit Clothed humans Opti- mized via Normal integration. CVPR, 2023. 1, 2, 5, 6

    2023

  53. [61]

    Depth any video with scalable synthetic data

    Honghui Yang, Di Huang, Wei Yin, Chunhua Shen, Haifeng Liu, Xiaofei He, Binbin Lin, Wanli Ouyang, and Tong He. Depth any video with scalable synthetic data. arXiv preprint arXiv:2410.10815, 2024. 2, 3, 7

    2024 arXiv

  54. [62]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. CVPR, 2024. 1, 2

    2024

  55. [63]

    Depth any- thing v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2. arXiv:2406.09414, 2024. 1, 2

    2024 arXiv

  56. [64]

    Metric3d: Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. ICCV, 2023. 3

    2023

  57. [65]

    Function4d: Real-time human vol- umetric capture from very sparse consumer rgbd sensors

    Tao Yu, Zerong Zheng, Kaiwen Guo, Pengpeng Liu, Qiong- hai Dai, and Yebin Liu. Function4d: Real-time human vol- umetric capture from very sparse consumer rgbd sensors. CVPR, 2021. 5

    2021

  58. [66]

    GLAMR: Global occlusion-aware human mesh recovery with dynamic cameras

    Ye Yuan, Umar Iqbal, Pavlo Molchanov, Kris Kitani, and Jan Kautz. GLAMR: Global occlusion-aware human mesh recovery with dynamic cameras. CVPR, 2022. 2

    2022

  59. [67]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 3

    2023

  60. [68]

    Adding conditional control to text-to-image diffusion models.ICCV,

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models.ICCV,

  61. [69]

    IC-light github page, 2024

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. IC-light github page, 2024. 6

    2024

  62. [71]

    I2VGen-XL: High-quality image-to-video syn- thesis via cascaded diffusion models

    Shiwei* Zhang, Jiayu* Wang, Yingya* Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qing, Xiang Wang, Deli Zhao, and Jin- gren Zhou. I2VGen-XL: High-quality image-to-video syn- thesis via cascaded diffusion models. In arXiv:2311.04145,

    arXiv

  63. [72]

    Consistent depth of moving objects in video

    Zhoutong Zhang, Forrester Cole, Richard Tucker, William T Freeman, and Tali Dekel. Consistent depth of moving objects in video. ACM Transactions on Graphics (TOG) , 40(4):1– 12, 2021. 2

    2021

  64. [73]

    SIFU: Side- view conditioned implicit function for real-world usable clothed human reconstruction

    Zechuan Zhang, Zongxin Yang, and Yi Yang. SIFU: Side- view conditioned implicit function for real-world usable clothed human reconstruction. CVPR, 2024. 2

    2024

  65. [74]

    Bilateral refer- ence for high-resolution dichotomous image segmentation

    Peng Zheng, Dehong Gao, Deng-Ping Fan, Li Liu, Jorma Laaksonen, Wanli Ouyang, and Nicu Sebe. Bilateral refer- ence for high-resolution dichotomous image segmentation. CAAI Artificial Intelligence Research, 3:9150038, 2024. 12

    2024

  66. [75]

    PaMIR: Parametric model-conditioned implicit representa- tion for image-based human reconstruction

    Zerong Zheng, Tao Yu, Yebin Liu, and Qionghai Dai. PaMIR: Parametric model-conditioned implicit representa- tion for image-based human reconstruction. TPAMI, 2021. 2

    2021

  67. [76]

    Magicvideo: Efficient video generation with latent diffusion models

    Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11018, 2022. 3

    2022 arXiv

  68. [77]

    “1q ř ktPK,dtPDt ››dt´ dgt t ››2 RMSEplogq: b 1řpKt““1q ř ktPK,dtPDt ››log dt´ log dgt t ››2 δă thr: 1řpKt““1q ř ktPK,dtPDt Kt

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. ECCV, 2024. 14 11 Supplementary Material of GeoMan: Temporally Consistent Human Geometry Estim...

    2024

  69. [79]

    24 Figure S18

    While our approach produces detailed and high-quality geometry, we plan to explore lightweight models or alternative training strategies to enable higher-resolution generation. 24 Figure S18. GeoMan demonstrates generalizablity across diverse scenarios, including occlusions, l...

  70. [2024]

    2, 3, 7, 8, 12, 14, 18

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.