Pith. sign in

REVIEW 3 major objections 4 minor 99 references

DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read This paper claims that predicting two point maps from a single image—one placing each pixel in the camera's 3D frame and the other on a canonical rest-pose model—reduces monocular 3D shape and pose reconstruction to a pixel-labeling task…

desk verdict A genuinely new dual point-map representation with strong real-image correspondence results, but the headline pose claim is never directly measured and the 3D benchmark is self-created and amended post-publication. read the letter →

arxiv 2412.04464 v5 pith:UI6DSOEL submitted 2024-12-05 cs.CV

classification cs.CV
keywords dualpointmapsdeformableobjectreconstructionposeestimationcanonicalamodalmonocular3Dquadruped
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that monocular 3D shape and pose reconstruction of deformable objects reduces to predicting a pair of point maps from a single image: one map locating each pixel's 3D point in the camera's frame (the posed map $P$) and the other locating the same point on a canonical rest-pose version of the object (the canonical map $Q$). Because the pose is the deformation that takes the object from rest to the observed configuration, it is obtained directly as the difference $P-Q$ between the two maps. The authors show that this representation is easy for a neural network to predict, and that training purely on synthetic renderings of one or two rigged models per animal category generalizes to real images, improving on previous methods for keypoint transfer and 3D reconstruction of quadrupeds. They extend the point-map idea to amodal, layered maps so the complete shape is recovered even through self-occlusions. The paper's significance is that it turns a hard inverse problem into a pixel-labeling task.

What carries the argument

The central object is the pair of point maps $(P, Q)$ called DualPMs. $P$ assigns each pixel $u$ the 3D location $p$ of the visible surface point in the camera reference frame; $Q$ assigns the same pixel the location $q$ of the corresponding point on the object in a canonical rest pose. The pose is the difference field $P-Q$, so no separate pose regression is needed. For amodal completeness, the authors use a layered representation $(P^*, Q^*)$: each layer stores the next ray-object intersection behind the previous one, with an opacity map per layer, analogous to depth peeling. Learning is supervised with a self-calibrated $L^2$ loss with per-pixel confidence, and the canonical map $Q$ is predicted from features of a pretrained DINOv2-plus-Stable-Diffusion extractor, while the posed map $P$ is predicted conditional on $Q$ only, which the ablation shows improves out-of-distribution generalization.

What would settle it

A concrete test: train a DualPM model on a single synthetic horse model and evaluate on real images of a quadruped with a body plan far outside the horse template, such as a giraffe or a camel. If the canonical point map $Q$ fails to place anatomically matching points (for example, the knee joint) at consistent canonical coordinates across images, or if the predicted posed map $P$ no longer tracks the visible silhouette after the deformation $P-Q$ is applied, then the claimed category-level canonical space does not transfer. The paper's own appendix already documents failure on extreme viewpoints and shapes far from the training distribution, so the observable prediction is that these failures become systematic when the test category's topology differs from the training rig.

Watch

Extended reading notes

Core claim

The discovery is the DualPM representation itself: instead of predicting depth or a single 3D point map, the network predicts two point maps from the same image, $P$ in camera space and $Q$ in a canonical object space where the object is in a neutral rest pose. The deformation field $P-Q$ is then, by construction, the object's pose, and matching pixels across images reduces to matching canonical coordinates. A second contribution is the amodal layered point map, which associates each pixel with all ray-surface intersections, visible and occluded, so a full reconstruction is obtained despite self-occlusions. With this target, a convolutional network trained on roughly 30k synthetic renderings of one or two rigged 3D models per category outperforms prior methods on PASCAL VOC keypoint transfer (PCK@0.1) and on the Animodel-Points Chamfer distance benchmarks, and generalizes to real images of horses, cows, sheep, and even unseen categories.

Load-bearing premise

The method assumes that one canonical rest-pose coordinate space can be shared by an entire object category, an anchor fixed by the one or two rigged 3D models used in training, and that the pretrained image features map any real instance into that same canonical space; the paper itself notes that aligning canonical spaces of multiple 3D models remains an unsolved problem, so if the canonical space is inconsistent across instances, $Q$ is ill-defined and $P-Q$ no longer encodes pose.

Editorial extensions

If this is right

  • Any geometric task for deformable objects—keypoint transfer, dense correspondence, scene flow between poses, skeleton fitting, and motion retargeting—reduces to reading off the two point maps, since pose is the difference $P-Q$ and correspondence is equality of canonical coordinates.
  • Training on one or two synthetic 3D models per category suffices to generalize to real images, outperforming self-supervised methods trained on large real-image collections such as 3D-Fauna and large-3D-asset methods such as Trellis on the reported benchmarks.
  • The amodal layered representation recovers the complete 3D shape of the object through self-occlusions, a capability that single-layer point maps lack.
  • The canonical point map $Q$ doubles as a useful image feature: conditioning $P$ on $Q$ rather than on raw image features improves out-of-distribution generalization, per the ablation study.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the pose is read out as a per-pixel difference field, the same representation should transfer to other articulated categories such as humans, hands, or birds without architectural changes, provided a rigged 3D model defines the category's canonical space.
  • The unsolved canonical-space alignment across multiple 3D models (acknowledged in App. B) suggests that the method's scalability depends on learning a shared canonical space; a testable direction is to fit DualPMs jointly to multiple rigs via a learned warp, which would let training data grow beyond one or two models.
  • The paper predicts only the expectation over unseen, self-occluded regions; a natural extension is to predict a distribution or multiple hypotheses per layer, which would address the unrealistic reconstructions the authors document for invisible parts.
  • The zero-shot generalization from a horse-trained model to cows and sheep hints that quadrupeds share a sufficiently aligned canonical structure; this suggests a testable scaling law: adding a few more rigged species should extend the model to any quadruped, and possibly to any articulated object, without retraining.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces DualPM, a representation for monocular 3D reconstruction of deformable objects: for each pixel, the network predicts a posed point map P in camera space and a canonical point map Q in a rest-pose object frame, with the pose encoded as the flow P − Q. It also extends point maps to amodal reconstruction via layered outputs that capture occluded surfaces. Training is done purely on synthetic renderings from one or two rigged 3D models per category from Animodel, using DINOv2 and Stable Diffusion features for the canonical predictor and conditioning the posed predictor on Q. Experiments report PCK on PASCAL VOC, Chamfer distance on the newly introduced Animodel-Points benchmark, ablations on conditioning and layer count, and qualitative skeleton fitting and animation results.

Significance. The representation is clearly formulated and the reported quantitative gains are substantial: PCK@0.1 on PASCAL VOC horses improves from 53.9 for 3D-Fauna to 73.2, and Chamfer distances on the introduced benchmark are much lower than existing methods. If the pose-decoding claim were validated, DualPM would be a simple and network-friendly target for deformable object reconstruction, and the synthetic-to-real generalization story would be notable. The paper also gives useful ablations showing the benefit of conditioning P on Q rather than on image features. However, the central pose claim is not yet empirically demonstrated, and the main 3D evaluation is partly based on a benchmark constructed by the authors from the same data family used for training. The independent PASCAL VOC result supports correspondence quality, but not the quantitative pose interpretation of P − Q.

major comments (3)
  1. [Sec. 3.1 and Fig. 1; Sec. 4.2–4.3] The paper defines the deformation field P − Q as the pose, but no experiment measures whether P − Q agrees with ground-truth articulation. Table 1 reports PCK, which evaluates the canonical map Q through 2D correspondences, and Chamfer distance of the posed point map P, which measures shape. Sec. 4.3 presents skeleton fitting only qualitatively (Figs. 1 and 6) and reports no joint-angle or bone-transform errors. Because the central claim is that 3D pose estimation reduces to DualPM prediction, the absence of any pose-error metric is a load-bearing gap. I would expect at least an evaluation on held-out synthetic poses from a rigged model where ground-truth joint rotations are known, reporting per-bone rotation error or a comparable pose metric, separately for seen and unseen templates.
  2. [App. B; Sec. 3.1] The interpretation of P − Q as pose relies on Q being defined in a single canonical space shared by all instances. App. B states that aligning canonical spaces across additional 3D models is unsolved, and training uses one or two models per category. For a novel instance such as a foal or donkey (Fig. 5), Q is anchored to the training template's rest pose, so P − Q contains body-shape and identity differences in addition to articulation. The paper does not isolate pose from shape. A quantitative experiment comparing predicted P − Q against an instance-specific canonical rest pose, or another explicit decomposition of shape and pose, is needed to support the pose claim; until then, the phrase 'shape and pose reconstruction' overstates what is measured.
  3. [App. D; Tab. 1] The main 3D quantitative benchmark, Animodel-Points, is introduced by this paper and is derived from the same Animodel data family used for training. This limits the strength of the cross-method shape-reconstruction comparison: the test distribution is close to the training distribution, and several baselines are the authors' own methods. The PASCAL VOC PCK result is independent and supports the correspondence/canonical-map claim, but it does not measure 3D shape accuracy. I recommend either re-running the shape comparison on an external 3D dataset or clearly separating the self-created benchmark results from the claim of general 3D reconstruction superiority; as it stands, the headline 'outperforms prior methods for 3D reconstruction' rests partly on a benchmark constructed from the training distribution.
minor comments (4)
  1. [Sec. 4.4] The first sentence says the ablation results are reported 'in Sec. 4.4,' which is the current section; it should point to Table 2.
  2. [Sec. 4.2] The inline reference 'Tab. 11' appears to be a typo for Table 1, and the footnote marker is rendered confusingly in the sentence 'The performance of our method and others is reported in Tab. 11'.
  3. [Sec. 3.2] The opacity loss Lσ is mentioned but its exact form is not given; since the layered amodal supervision is a contribution, please specify the loss explicitly.
  4. [References] The same work 'MagicPony' appears to be cited as both [70] and [72] in the text; please make the citation consistent or clarify that these are distinct versions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DualPM pose reduction is an explicit representation definition rather than a hidden reuse of fitted inputs, and the main generalization claims are grounded in external PASCAL VOC evaluations.

full rationale

The paper's central construction is a representation reduction: pose is defined as the deformation field P−Q (Sec. 1: 'the deformation field that we wish to recover is simply the difference between the two point maps'), so the claim that pose estimation reduces to DualPM prediction follows from the definition. This is a design choice rather than a circular derivation, because P and Q are image-conditioned network outputs trained against rendered ground-truth point maps, not quantities refit to the evaluation target. The key empirical claim, that networks trained on one or two synthetic models generalize to real images, is supported by the external PASCAL VOC PCK@0.1 benchmark, which is independent of the authors' training data and evaluation pipeline. The Animodel-Points benchmark (App. D) is derived from the authors' Farm3D data and several baselines (Farm3D, 3D-Fauna, MagicPony) are from the same group, but this is ordinary use of published resources rather than load-bearing circularity; the benchmark explicitly excludes training poses. App. B's admission that canonical-space alignment across multiple 3D models is unsolved is a validity limitation for the pose interpretation on novel identities, not a circular step, since no fitted parameter is renamed as a prediction. The absence of a direct quantitative pose-accuracy metric is a correctness and validation gap, not a circularity of the derivation chain.

Assumptions & free parameters 5 free parameters · 4 assumptions · 2 invented entities

The central claim rests on four domain assumptions: pose-invariant pretrained features, a consistent category-level canonical space, even ray intersections, and available segmentation masks. The main hand-chosen parameters are the number of amodal layers, the confidence-loss weight, feature dimension, output resolution, and the training schedule. The paper introduces two new representations (DualPMs and layered amodal point maps) and validates them empirically; they are treated as invented entities with independent evidence because the paper demonstrates prediction and downstream utility.

free parameters (5)
  • number of predicted layers K = 4 (4 ray pairs, 8 layers)
    Set to 4 in App. C; ablation shows minimal change beyond 4 on Animodel, but this caps the completeness of amodal reconstruction.
  • confidence loss log-weight alpha = not reported
    Constant in the self-calibrated L2 loss of Sec. 3.1; value is not given, but it balances confidence prediction against point-map accuracy.
  • feature dimension after PCA = 64
    App. C reduces DINOv2+StableDiffusion features to 64 dimensions before predicting Q.
  • training schedule = 100k steps, batch size 12, LR 6e-4, step decay 30k with factor 0.5
    Hand-selected hyperparameters in App. C needed to reproduce training.
  • point map resolution = 160x160
    Output resolution of layered point maps in App. C.
assumptions (4)
  • domain assumption Pre-trained DINOv2 and Stable Diffusion features, combined as in [86], provide pose- and deformation-invariant features that serve as a proxy for the canonical point map.
    Sec. 3.1 conditions the canonical predictor on F = Ψ(I); the method relies on these features transferring from synthetic renders to real images.
  • domain assumption A single canonical space can be defined for an entire object category, anchored by the one or two rigged models used for training.
    The ground-truth Q is computed in the training model's rest-pose space; App. B states that adding more models requires solving canonical-space alignment, which is not addressed.
  • domain assumption The camera is always outside the object, so every viewing ray intersects the surface an even number of times.
    Sec. 3.2 relies on even-length intersection sequences to form point pairs (p1,p2),(p3,p4),... in the amodal representation.
  • domain assumption An accurate object segmentation mask M is available at inference time.
    Losses are computed only over mask pixels and the model consumes the mask; Fig. A9 shows that segmentation errors cause reconstruction failures.
invented entities (2)
  • Dual Point Maps (P, Q) independent evidence
    purpose: A pair of pixel-aligned 3D maps from one image: P gives posed camera-space coordinates and Q gives canonical rest-pose coordinates; the deformation field P-Q defines object pose.
    The paper shows the maps can be predicted by a network and used for reconstruction, keypoint transfer, skeleton fitting, and animation, so the concept is validated outside its definition.
  • Layered amodal point maps (P*, Q*) independent evidence
    purpose: An image-aligned stack of point maps that assigns each pixel all ray-object intersections, enabling reconstruction of self-occluded surfaces.
    Supervised with opacity maps and used in the main experiments; the ablation shows 4 layers give most of the benefit on Animodel-Points.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction." pith.science (2026). https://pith.science/paper/UI6DSOEL

@misc{pith2026241204464,
  author       = {Pith},
  title        = {Pith review of: DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UI6DSOEL}},
  note         = {Machine review of arXiv:2412.04464}
}
read the original abstract

The choice of data representation is a key factor in the success of deep learning in geometric tasks. For instance, DUSt3R recently introduced the concept of viewpoint-invariant point maps, generalizing depth prediction and showing that all key problems in the 3D reconstruction of static scenes can be reduced to predicting such point maps. In this paper, we develop an analogous concept for a very different problem: the reconstruction of the 3D shape and pose of deformable objects. To this end, we introduce Dual Point Maps (DualPM), where a pair of point maps is extracted from the same image-one associating pixels to their 3D locations on the object and the other to a canonical version of the object in its rest pose. We also extend point maps to amodal reconstruction to recover the complete shape of the object, even through self-occlusions. We show that 3D reconstruction and 3D pose estimation can be reduced to the prediction of DualPMs. Empirically, we demonstrate that this representation is a suitable target for deep networks to predict. Specifically, we focus on modeling quadrupeds, showing that DualPMs can be trained purely on synthetic 3D data, consisting of one or two models per category, while generalizing effectively to real images. With this approach, we achieve significant improvements over previous methods for the 3D analysis and reconstruction of such objects.

Figures

Figures reproduced from arXiv: 2412.04464 by the authors.

Figure 1
Figure 1. Left: We map an image of an object to its Dual Point Maps (DualPMs), a pair of point maps P, defined in a camera space, and Q, defined in a canonical space where the object has a neutral pose. The pose is thus given by the flow P − Q. Right: The DualPMs are easy to predict with a neural network, enabling effective 3D object reconstruction and facilitating geometric tasks like detecting 3D keypoints and fitting a 3D … view at source ↗
Figure 2
Figure 2. An amodal point map associates each pixel u to an even number of objects points pi , corresponding to the locations where the pixel’s ray intersects the object’s surface. Predicting an amodal point map reconstructs the entire object despite self-occlusions. recover the pose of the object: in fact, the deformation field that we wish to recover is simply the difference between the two point maps. Knowledge of this def… view at source ↗
Figure 3
Figure 3. Method overview. We preprocess the input image I by obtaining a segmentation mask M and image features F using pretrained networks. Subsequently, we predict the layered canonical point map {Q ∗ i } 2K i=1. Conditioned on {Q ∗ i } 2K i=1, we predict the layered posed point map {P ∗ i } 2K i=1, along with the associated opacity maps {σ ∗ i } 2K i=3. Both layered point maps are then decoded to produce the canonical poi… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Synthetic training data. We generate synthetic training data by rendering a rigged 3D model of a category in various poses with different environmental maps and under random viewpoints. mesh is then rendered from the camera’s viewpoint. Dur￾ing the rendering process, t…
Figure 5
Figure 5. Figure 5: Comparison with the state-of-the-art. We compare our method with 3D-Fauna [33]. For visualization, we paint each point of the recovered point map P with its canonical coordinate. Our reconstructed shape aligns more closely with the object’s structure compared to 3D-Fau…
Figure 6
Figure 6. Figure 6: Animation. DualPMs allow for fitting a 3D skeleton, which can subsequently be used to animate the 3D reconstruction by utilizing existing animations through motion retargeting. used in this benchmark. The performance of our method and others is reported in Tab. 1 1 Our…
Figure 7
Figure 7. Figure 7: Effect of different conditioning schemes. Conditioning solely on the canonical point maps Q, as opposed to using both the image features F and the canonical point maps Q, results in better generalization, particularly in extreme out-of-distribution images. can be easil…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

99 extracted references · 57 canonical work pages

  1. [1]

    AutoAvatar: Au- toregressive neural fields for dynamic avatar modeling

    Ziqian Bai, Timur Bagautdinov, Javier Romero, Michael Zollh¨ofer, Ping Tan, and Shunsuke Saito. AutoAvatar: Au- toregressive neural fields for dynamic avatar modeling. In Proc. ECCV, 2022

  2. [2]

    Narasimhan

    Aayush Bansal, Minh V o, Yaser Sheikh, Deva Ramanan, and Srinivasa G. Narasimhan. 4D visualization of dynamic events from unconstrained multi-view videos. In Proc. CVPR, 2020

  3. [3]

    Fitzgibbon, and Roberto Cipolla

    Benjamin Biggs, Thomas Roddick, Andrew W. Fitzgibbon, and Roberto Cipolla. Creatures great and SMAL: recovering the shape and motion of animals from video. In Proc. ECCV, 2018

  4. [4]

    Goat image dataset

    Masum Billah, Yu Jiantao, and Yu Jiang. Goat image dataset. Mendeley Data, V2, 2020

  5. [5]

    Hexplane: A fast representation for dynamic scenes

    Ang Cao and Justin Johnson. Hexplane: A fast representation for dynamic scenes. arXiv.cs, abs/2301.09632, 2023

  6. [6]

    Relighting4D: Neural relightable human from videos

    Zhaoxi Chen and Ziwei Liu. Relighting4D: Neural relightable human from videos. In Proc. ECCV, 2022

  7. [7]

    Dream- Scene4D: dynamic multi-object scene generation from monocular videos

    Wen-Hsuan Chu, Lei Ke, and Katerina Fragkiadaki. Dream- Scene4D: dynamic multi-object scene generation from monocular videos. arXiv, 2405.02280, 2024

  8. [8]

    Tenen- baum, and Jiajun Wu

    Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B. Tenen- baum, and Jiajun Wu. Neural radiance flow for 4d view synthesis and video processing. In Proc. ICCV, 2021

Show all 99 references
  1. [9]

    MASt3R- SfM: a fully-integrated solution for unconstrained structure- from-motion

    Bardienus Duisterhof, Lojze Zust, Philippe Weinzaepfel, Vin- cent Leroy, Yohann Cabon, and Jerome Revaud. MASt3R- SfM: a fully-integrated solution for unconstrained structure- from-motion. arXiv, 2409.19152, 2024

  2. [10]

    Duisterhof, Zhao Mandi, Yunchao Yao, Jia-Wei Liu, Mike Zheng Shou, Shuran Song, and Jeffrey Ichnowski

    Bardienus P. Duisterhof, Zhao Mandi, Yunchao Yao, Jia-Wei Liu, Mike Zheng Shou, Shuran Song, and Jeffrey Ichnowski. MD-Splatting: Learning metric deformation from 4d gaus- sians in highly deformable scenes. arXiv.cs, 2312.00583, 2023

  3. [11]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bj¨orn Ommer. Taming transformers for high-resolution image synthesis. In Proc. CVPR, 2021

  4. [12]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. IJCV, 88(2):303–338, 2010

  5. [13]

    Gaussian splashing: Dynamic fluid synthesis with gaussian splatting

    Yutao Feng, Xiang Feng, Yintong Shang, Ying Jiang, Chang Yu, Zeshun Zong, Tianjia Shao, Hongzhi Wu, Kun Zhou, Chenfanfu Jiang, and Yin Yang. Gaussian splashing: Dynamic fluid synthesis with gaussian splatting. arXiv.cs, 2401.15318, 2024

  6. [14]

    K-planes: Explicit radiance fields in space, time, and appearance

    Sara Fridovich-Keil, Giacomo Meanti, Frederik Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. arXiv.cs, abs/2301.10241, 2023

  7. [15]

    Dynamic neural radiance fields for monocular 4d facial avatar reconstruction

    Guy Gafni, Justus Thies, Michael Zollh ¨ofer, and Matthias Nießner. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. In Proc. CVPR, 2021

  8. [16]

    Dynamic view synthesis from dynamic monocular video

    Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. Dynamic view synthesis from dynamic monocular video. In Proc. ICCV, 2021

  9. [17]

    Shape and viewpoints without keypoints

    Shubham Goel, Angjoo Kanazawa, and Jitendra Malik. Shape and viewpoints without keypoints. In Proc. ECCV, 2020

  10. [18]

    Farm3D: Learning articulated 3D ani- mals by distilling 2D diffusion

    Tomas Jakab, Ruining Li, Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi. Farm3D: Learning articulated 3D ani- mals by distilling 2D diffusion. In Proc. 3DV, 2024

  11. [19]

    VR-GS: A physical dynamics-aware interactive gaussian splatting system in virtual reality

    Ying Jiang, Chang Yu, Tianyi Xie, Xuan Li, Yutao Feng, Huamin Wang, Minchen Li, Henry Lau, Feng Gao, Yin Yang, and Chenfanfu Jiang. VR-GS: A physical dynamics-aware interactive gaussian splatting system in virtual reality. arXiv, 2401.16663, 2024

  12. [20]

    What uncertainties do we need in Bayesian deep learning for computer vision? Proc

    Alex Kendall and Yarin Gal. What uncertainties do we need in Bayesian deep learning for computer vision? Proc. NeurIPS, 2017

  13. [21]

    3D Gaussian Splatting for real-time radi- ance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk¨uhler, and George Drettakis. 3D Gaussian Splatting for real-time radi- ance field rendering. Proc. SIGGRAPH, 42(4), 2023

  14. [22]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Proc. ICLR, 2015

  15. [23]

    Berg, Wan-Yen Lo, Piotr Doll´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll´ar, and Ross Girshick. Segment anything. In Proc. CVPR, 2023

  16. [24]

    DynMF: Neural motion factorization for real-time dynamic view synthesis with 3D Gaussian splatting

    Agelos Kratimenos, Jiahui Lei, and Kostas Daniilidis. DynMF: Neural motion factorization for real-time dynamic view synthesis with 3D Gaussian splatting. arXiv.cs, abs/2312.00112, 2023

  17. [25]

    Canonical surface mapping via geometric cycle consistency

    Nilesh Kulkarni, Abhinav Gupta, and Shubham Tulsiani. Canonical surface mapping via geometric cycle consistency. In Proc. ICCV, 2019

  18. [26]

    Fouhey, and Shub- ham Tulsiani

    Nilesh Kulkarni, Abhinav Gupta, David F. Fouhey, and Shub- ham Tulsiani. Articulation-aware canonical surface mapping. In Proc. CVPR, pages 449–458, 2020

  19. [27]

    Monocular dense 3D reconstruction of a complex dynamic scene from two perspective frames

    Suryansh Kumar, Yuchao Dai, and Hongdong Li. Monocular dense 3D reconstruction of a complex dynamic scene from two perspective frames. In Proc. ICCV, 2017

  20. [28]

    MoSca: dynamic gaussian fusion from casual videos via 4d motion scaffolds

    Jiahui Lei, Yijia Weng, Adam Harley, Leonidas Guibas, and Kostas Daniilidis. MoSca: dynamic gaussian fusion from casual videos via 4d motion scaffolds. arXiv, 2405.17421, 2024

  21. [29]

    Dessie: Disentanglement for articulated 3D horse shape and pose estimation from images

    Ci Li, Yi Yang, Zehang Weng, Elin Hernlund, Silvia Zuffi, and Hedvig Kjellstr¨om. Dessie: Disentanglement for articulated 3D horse shape and pose estimation from images. arXiv, 2410.03438, 2024

  22. [30]

    Self-supervised single-view 3D reconstruction via semantic consistency

    Xueting Li, Sifei Liu, Kihwan Kim, Shalini De Mello, Varun Jampani, Ming-Hsuan Yang, and Jan Kautz. Self-supervised single-view 3D reconstruction via semantic consistency. In Proc. ECCV, 2020

  23. [31]

    Neural scene flow fields for space-time view synthesis of dynamic scenes

    Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. In Proc. CVPR, 2021

  24. [32]

    DynIBaR: Neural dynamic image-based rendering

    Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, and Noah Snavely. DynIBaR: Neural dynamic image-based rendering. In Proc. CVPR, 2023. 9

  25. [33]

    Learning the 3D fauna of the Web

    Zizhang Li, Dor Litvak, Ruining Li, Yunzhi Zhang, Tomas Jakab, Christian Rupprecht, Shangzhe Wu, Andrea Vedaldi, and Jiajun Wu. Learning the 3D fauna of the Web. In Proc. CVPR, 2024

  26. [34]

    GauFRe: Gaussian deformation fields for real-time dynamic novel view synthesis

    Yiqing Liang, Numair Khan, Zhengqin Li, Thu Nguyen- Phuoc, Douglas Lanman, James Tompkin, and Lei Xiao. GauFRe: Gaussian deformation fields for real-time dynamic novel view synthesis. arXiv.cs, abs/2312.11458, 2023

  27. [35]

    Im4d: High-fidelity and real- time novel view synthesis for dynamic scenes

    Haotong Lin, Sida Peng, Zhen Xu, Tao Xie, Xingyi He, Hu- jun Bao, and Xiaowei Zhou. Im4d: High-fidelity and real- time novel view synthesis for dynamic scenes. In Proc. SIG- GRAPH Asia, 2023

  28. [36]

    Gaussian- flow: 4d reconstruction with dynamic 3d gaussian particle

    Youtian Lin, Zuozhuo Dai, Siyu Zhu, and Yao Yao. Gaussian- flow: 4d reconstruction with dynamic 3d gaussian particle. In Proc. CVPR, 2024

  29. [37]

    Align your gaussians: Text-to-4d with dynamic 3d gaussians and composed diffusion models

    Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler, and Karsten Kreis. Align your gaussians: Text-to-4d with dynamic 3d gaussians and composed diffusion models. In Proc. CVPR, 2024

  30. [38]

    Dynamic gaussians mesh: Consistent mesh reconstruction from monocular videos

    Isabella Liu, Hao Su, and Xiaolong Wang. Dynamic gaussians mesh: Consistent mesh reconstruction from monocular videos. arXiv, 2404.12379, 2024

  31. [39]

    Neural human video rendering: Joint learning of dynamic textures and rendering-to-video transla- tion

    Lingjie Liu, Weipeng Xu, Marc Habermann, Michael Zollh¨ofer, Florian Bernard, Hyeongwoo Kim, Wenping Wang, and Christian Theobalt. Neural human video rendering: Joint learning of dynamic textures and rendering-to-video transla- tion. arXiv.cs, abs/2001.04947, 2020

  32. [40]

    Saragih, Gabriel Schwartz, Andreas M

    Stephen Lombardi, Tomas Simon, Jason M. Saragih, Gabriel Schwartz, Andreas M. Lehrmann, and Yaser Sheikh. Neural volumes: learning dynamic renderable volumes from images. ACM Trans. Graph., 38(4), 2019

  33. [41]

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: a skinned multi- person linear model. ACM Trans. on Graphics (TOG), 2015

  34. [42]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthe- sis. In Proc. ECCV, 2020

  35. [43]

    Newcombe, Dieter Fox, and Steven M

    Richard A. Newcombe, Dieter Fox, and Steven M. Seitz. DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proc. CVPR, 2015

  36. [44]

    Capturing the geometry of object categories from video supervision

    David Novotn´y, Diane Larlus, and Andrea Vedaldi. Capturing the geometry of object categories from video supervision. 2018

  37. [45]

    Maxime Oquab, Timoth´ee Darcet, Th´eo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido As- sran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po- Yao Huang, Shang-Wen Li, Ishan Misra, Michae...

  38. [46]

    Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans

    Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In Proc. CVPR, 2021

  39. [47]

    D-NeRF: Neural radiance fields for dynamic scenes

    Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-NeRF: Neural radiance fields for dynamic scenes. In Proc. CVPR, 2021

  40. [48]

    L4GM: Large 4D Gaussian reconstruction model

    Jiawei Ren, Kevin Xie, Ashkan Mirzaei, Hanxue Liang, Xiao- hui Zeng, Karsten Kreis, Ziwei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, and Huan Ling. L4GM: Large 4D Gaussian reconstruction model. arXiv, 2406.10324, 2024

  41. [49]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proc. CVPR, 2022

  42. [50]

    Barc: Learning to regress 3d dog shape from images by exploiting breed information

    Nadine R¨uegg, Silvia Zuffi, Konrad Schindler, and Michael J Black. Barc: Learning to regress 3d dog shape from images by exploiting breed information. In Proc. CVPR, 2022

  43. [51]

    Video pop-up: Monocular 3D reconstruction of dynamic scenes

    Chris Russell, Rui Yu, and Lourdes Agapito. Video pop-up: Monocular 3D reconstruction of dynamic scenes. In Proc. ECCV, 2014

  44. [52]

    Freeman, Fr´edo Durand, Joshua B

    Prafull Sharma, Ayush Tewari, Yilun Du, Sergey Zakharov, Rares Ambrus, Adrien Gaidon, William T. Freeman, Fr´edo Durand, Joshua B. Tenenbaum, and Vincent Sitzmann. Seeing 3d objects in a single image via self-supervised static-dynamic disentanglement. arXiv.cs, abs/2207.11232, 2022

  45. [53]

    Learning dynamic facial radiance fields for few-shot talking head synthesis

    Shuai Shen, Wanhua Li, Zheng Zhu, Yueqi Duan, Jie Zhou, and Jiwen Lu. Learning dynamic facial radiance fields for few-shot talking head synthesis. In Proc. ECCV, 2022

  46. [54]

    NeRF- Player: A streamable dynamic scene representation with de- composed neural radiance fields

    Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas Geiger. NeRF- Player: A streamable dynamic scene representation with de- composed neural radiance fields. IEEE Trans. on Visualiza- tion and Computer Graphics, 29(5), 2023

  47. [55]

    Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Proc. ICLR, 2021

  48. [56]

    Dynamic Gaussian marbles for novel view synthesis of casual monocular videos

    Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, and Leonidas Guibas. Dynamic Gaussian marbles for novel view synthesis of casual monocular videos. arXiv, 2406.18717, 2024

  49. [57]

    Unsuper- vised learning of object frames by dense equivariant image labelling

    James Thewlis, Hakan Bilen, and Andrea Vedaldi. Unsuper- vised learning of object frames by dense equivariant image labelling. In Proc. NeurIPS, 2017

  50. [58]

    Unsu- pervised learning of object landmarks by factorized spatial embeddings

    James Thewlis, Hakan Bilen, and Andrea Vedaldi. Unsu- pervised learning of object landmarks by factorized spatial embeddings. In Proc. ICCV, 2017

  51. [59]

    Modelling and unsupervised learning of symmetric deformable object categories

    James Thewlis, Hakan Bilen, and Andrea Vedaldi. Modelling and unsupervised learning of symmetric deformable object categories. In Proc. NeurIPS, 2018

  52. [60]

    He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, and Leonidas J. Guibas. Normalized object coordinate space for category-level 6d object pose and size estimation. In Proc. CVPR, pages 2642–2651, 2019

  53. [61]

    Mor- pheus: Neural dynamic 360 ° surface reconstruction from monocular RGB-D video

    Hengyi Wang, Jingwen Wang, and Lourdes Agapito. Mor- pheus: Neural dynamic 360 ° surface reconstruction from monocular RGB-D video. arXiv.cs, abs/2312.00778, 2023

  54. [62]

    PoseDiffusion: solving pose estimation via diffusion-aided bundle adjustment

    Jianyuan Wang, Christian Rupprecht, and David Novotny. PoseDiffusion: solving pose estimation via diffusion-aided bundle adjustment. In Proc. ICCV, 2023. 10

  55. [63]

    VGGSfM: visual geometry grounded deep structure from motion

    Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. VGGSfM: visual geometry grounded deep structure from motion. In Proc. CVPR, 2024

  56. [64]

    Fourier PlenOctrees for dynamic radiance field rendering in real-time

    Liao Wang, Jiakai Zhang, Xinhang Liu, Fuqiang Zhao, Yan- shun Zhang, Yingliang Zhang, Minye Wu, Jingyi Yu, and Lan Xu. Fourier PlenOctrees for dynamic radiance field rendering in real-time. In Proc. CVPR, 2022

  57. [65]

    Shape of motion: 4D reconstruc- tion from a single video

    Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4D reconstruc- tion from a single video. arXiv, 2407.13764, 2024

  58. [66]

    DUSt3R: Geometric 3D vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. DUSt3R: Geometric 3D vision made easy. In Proc. CVPR, 2024

  59. [67]

    MonoRec: Semi-supervised dense reconstruction in dynamic environments from a sin- gle moving camera

    Felix Wimbauer, Nan Yang, Lukas von Stumberg, Niclas Zeller, and Daniel Cremers. MonoRec: Semi-supervised dense reconstruction in dynamic environments from a sin- gle moving camera. In Proc. CVPR, 2021

  60. [68]

    4D gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4D gaussian splatting for real-time dynamic scene rendering. In Proc. CVPR, 2023

  61. [69]

    Un- supervised learning of probably symmetric deformable 3D objects from images in the wild

    Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi. Un- supervised learning of probably symmetric deformable 3D objects from images in the wild. In Proc. CVPR, 2020

  62. [70]

    Magicpony: Learning articulated 3d animals in the wild

    Shangzhe Wu, Ruining Li, Tomas Jakab, Christian Rupprecht, and Andrea Vedaldi. Magicpony: Learning articulated 3d animals in the wild. arXiv.cs, 2022

  63. [71]

    DOVE: Learning deformable 3D objects by watching videos

    Shangzhe Wu, Tomas Jakab, Christian Rupprecht, and Andrea Vedaldi. DOVE: Learning deformable 3D objects by watching videos. In IJCV, 2023

  64. [72]

    MagicPony: Learning articulated 3D animals in the wild

    Shangzhe Wu, Ruining Li, Tomas Jakab, Christian Rupprecht, and Andrea Vedaldi. MagicPony: Learning articulated 3D animals in the wild. In Proc. CVPR, 2023

  65. [73]

    Structured 3D latents for scalable and versatile 3D generation

    Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3D latents for scalable and versatile 3D generation. arXiv, 2412.01506, 2024

  66. [74]

    PhysGaussian: Physics- integrated 3D Gaussians for generative dynamics

    Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. PhysGaussian: Physics- integrated 3D Gaussians for generative dynamics. arXiv.cs, abs/2311.12198, 2023

  67. [75]

    Street gaus- sians for modeling dynamic urban scenes

    Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, and Xianpeng Lang. Street gaus- sians for modeling dynamic urban scenes. arXiv, 2024

  68. [76]

    Freeman, and Ce Liu

    Gengshan Yang, Deqing Sun2, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T. Freeman, and Ce Liu. LASR: Learning articulated shape reconstruction from a monocular video. In Proc. CVPR, 2021

  69. [77]

    BANMo: Building ani- matable 3d neural models from many casual videos

    Gengshan Yang, Minh V o, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, and Hanbyul Joo. BANMo: Building ani- matable 3d neural models from many casual videos. In Proc. CVPR, 2022

  70. [78]

    Dinesh Reddy, and Deva Ramanan

    Gengshan Yang, Chaoyang Wang, N. Dinesh Reddy, and Deva Ramanan. Reconstructing animatable categories from videos. In Proc. CVPR, 2023

  71. [79]

    Depth anything V2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything V2. arXiv, 2406.09414, 2024

  72. [80]

    Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting

    Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. In Proc. ICLR, 2024

  73. [81]

    LASSIE: learning articulated shapes from sparse image ensemble via 3d part discovery

    Chun-Han Yao, Wei-Chih Hung, Yuanzhen Li, Michael Ru- binstein, Ming-Hsuan Yang, and Varun Jampani. LASSIE: learning articulated shapes from sparse image ensemble via 3d part discovery. arXiv.cs, abs/2207.03434, 2022

  74. [82]

    ARTIC3D: Learning robust articulated 3D shapes from noisy web image collections

    Chun-Han Yao, Amit Raj, Wei-Chih Hung, Yuanzhen Li, Michael Rubinstein, Ming-Hsuan Yang, and Varun Jampani. ARTIC3D: Learning robust articulated 3D shapes from noisy web image collections. Proc. NeurIPS, 2023

  75. [83]

    4DGen: Grounded 4D content generation with spatial-temporal consistency

    Yuyang Yin1, Dejia Xu2, Zhangyang Wang, Yao Zhao, and Yunchao Wei. 4DGen: Grounded 4D content generation with spatial-temporal consistency. arXiv.cs, 2023

  76. [84]

    Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera

    Jae Shin Yoon, Kihwan Kim, Orazio Gallo, Hyun Soo Park, and Jan Kautz. Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera. In Proc. CVPR, 2020

  77. [85]

    EgoGaussian: Dynamic scene understanding from egocentric video with 3D gaussian splatting

    Daiwei Zhang, Gengyan Li, Jiajie Li, Micka ¨el Bressieux, Otmar Hilliges, Marc Pollefeys, Luc Van Gool, and Xi Wang. EgoGaussian: Dynamic scene understanding from egocentric video with 3D gaussian splatting. arXiv.cs, abs/2406.19811, 2024

  78. [86]

    A tale of two features: Stable diffusion complements DINO for zero-shot semantic correspondence

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Pola- nia Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements DINO for zero-shot semantic correspondence. arXiv.cs, abs/2305.15347, 2023

  79. [87]

    MonST3R: a simple approach for estimating geometry in the presence of motion

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. MonST3R: a simple approach for estimating geometry in the presence of motion. arXiv, 2410.03825, 2024

  80. [88]

    Animate124: Animating one image to 4D dynamic scene

    Yuyang Zhao, Zhiwen Yan, Enze Xie, Lanqing Hong, Zhen- guo Li, and Gim Hee Lee. Animate124: Animating one image to 4D dynamic scene. arXiv.cs, abs/2311.14603, 2023

  81. [89]

    Jacobs, and Michael J

    Silvia Zuffi, Angjoo Kanazawa, David W. Jacobs, and Michael J. Black. 3D menagerie: Modeling the 3D shape and pose of animals. In Proc. CVPR, 2017

  82. [90]

    Lions and tigers and bears: Capturing non-rigid, 3d, articulated shape from images

    Silvia Zuffi, Angjoo Kanazawa, and Michael J Black. Lions and tigers and bears: Capturing non-rigid, 3d, articulated shape from images. In Proc. CVPR, 2018

  83. [91]

    Varen: Very accurate and realistic equine network

    Silvia Zuffi, Ylva Mellbin, Ci Li, Markus Hoeschle, Hedvig Kjellstr¨om, Senya Polikovsky, Elin Hernlund, and Michael J Black. Varen: Very accurate and realistic equine network. In Proc. CVPR, pages 5374–5383, 2024. 11 Appendix A. Generalization to unseen categories Given the g...

  84. [92]

    Coordinate transformation: The source mesh is trans- formed into model-view coordinates

  85. [93]

    Uniform scaling: The mesh is uniformly scaled by a fac- tor of V −1/3, where V is the mesh volume, to normalize its size

  86. [94]

    Evaluation protocol For a given generated point cloud, the following evaluation steps are performed:

    Point sampling: 20,000 points are uniformly sampled from the surface of the scaled mesh to serve as the ground truth target. Evaluation protocol For a given generated point cloud, the following evaluation steps are performed:

  87. [95]

    Resampling: If the input point cloud does not contain 20,000 points, it is resampled to this size

  88. [96]

    Rotational ambiguity handling: The 20,000 point sam- ple is duplicated, and the duplicate is rotated 180 degrees around its vertical axis

  89. [97]

    Evaluation on Animodel

    Subsampling for fitting: A subset of 10,000 points is sampled from the input to be used in the alignment pro- 13 Method Chamfer Distance (cm) Real-Sized Normalized Horse Cow Sheep Horse Cow Sheep A-CSM [26] 7.60 ± 3.07 6.71 ± 1.81 2.84 ± 0.77 2.73 ± 1.13 2.35 ± 0.68 2.48 ± 0.7...

  90. [98]

    The process runs for a maxi- mum of 200 steps or until convergence, using the MSE bidirectional Chamfer distance as the objective function

    Alignment: Both the original and rotated samples are aligned to the ground truth target using the Iterative Clos- est Point (ICP) algorithm. The process runs for a maxi- mum of 200 steps or until convergence, using the MSE bidirectional Chamfer distance as the objective function

  91. [99]

    Chamfer distance performance is evaluated by providing scale, rotation, and translation degrees of freedom to the transformation estimation

    Final score: The lower of the two costs from the align- ment steps is reported as the final score. Chamfer distance performance is evaluated by providing scale, rotation, and translation degrees of freedom to the transformation estimation. Model-view chamfer distance is evalua...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.