Unsupervised 2D-3D lifting of non-rigid objects using local constraints

Benjamin Graham; Lourdes Agapito; Shalini Maiti

REVIEW 3 major objections 7 minor 31 references

Unsupervised 2D-3D lifting of non-rigid objects using local constraints

T0 review · 3 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Applying low-rank constraints to local keypoint neighborhoods cuts non-rigid 3D reconstruction error by over 70%.

desk verdict The local-low-rank loss story is real and the S-Up3D improvement is not a metric artifact; the one-shot experiments and missing code are the soft spots. read the letter →

arxiv 2504.19227 v1 pith:KE5FESHP submitted 2025-04-27 cs.CV

Shalini Maiti , Lourdes Agapito , Benjamin Graham This is my paper · ORCID

classification cs.CV

keywords 2D-3Dliftingnon-rigidstructurefrommotionunsupervisedlearninglow-rankconstraintsMLP-Mixerocclusionlocalshapepriorsone-shotreconstruction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Non-rigid 3D reconstruction from 2D keypoints is ill-posed, especially under occlusion, and classical deep solutions embed global low-rank shape priors directly into the network architecture. This paper argues that a generic, high-capacity model trained with an unsupervised loss can outperform those specialized architectures, provided the low-rank prior is applied locally — to automatically selected neighborhoods of keypoints — rather than to the whole shape. The claimed result is a mean per-joint position error of 0.0163 on the S-Up3D benchmark, more than 70% below the best previous baseline, with a standard deviation of 0.0014 across five seeds. If true, the geometric knowledge needed for lifting lives in the loss function, not in a bespoke network design, which simplifies training and extends naturally to one-shot video reconstruction.

What carries the argument

The load-bearing mechanism is the subset loss: for each batch, it selects local neighborhoods of keypoints (by k-nearest neighbors in the predicted 3D space, or randomly), removes translation, aligns them to a batchwise mean shape via SVD and an optimal rigid alignment, and then penalizes the log Gramian determinant of the residual error matrix. Minimizing this log-volume of non-rigid variation pushes the network to predict body parts whose deformation is nearly low-rank. A second loss, the occlusion loss, clamps the cosine similarity between centered visibility and depth vectors at -0.05, resolving the depth mirror flip so the model does not suffer from z-flip ambiguity. The network itself is a generic MLP-Mixer with no built-in low-rank layers; the geometric prior lives entirely in the loss.

What would settle it

Train the same ALLRAP pipeline on a deformable object with locally independent motion — for example, a cloth or flag where nearby points move with many independent degrees of freedom even within small neighborhoods — and measure MPJPE against a global low-rank baseline. If the local subset loss does not degrade relative to baselines on such data, the claim that local low-rankness is the active prior would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that for non-rigid objects with a semi-dense set of keypoints, local shape coherence is a stronger and more trainable prior than global low-rankness. The paper proposes ALLRAP, a method that trains an MLP-Mixer — a transformer-like network without attention — to directly predict 3D coordinates from partially occluded 2D keypoints, using two unsupervised batch losses. The subset loss selects local neighborhoods of keypoints, aligns them across the batch by rigid rotations, and penalizes the log Gramian determinant of the residual error, i.e., the log-volume of non-rigid deformation left after alignment. The occlusion loss enforces a weak negative correlation between visibility and depth to fix the depth flip ambiguity. The paper reports that this combination reduces the state-of-the-art MPJPE on S-Up3D by over 70% and also produces strong one-shot reconstructions on animal and human motion sequences.

Load-bearing premise

The subset loss assumes that automatically selected local neighborhoods of keypoints correspond to coherent body parts whose 3D deformation is approximately low-rank after rigid alignment; if real deformations in a neighborhood have many independent degrees of freedom, the loss imposes a wrong prior.

Editorial extensions

If this is right

On the S-Up3D benchmark, ALLRAP reaches MPJPE 0.0163, a reduction of more than 70% over the best previous baseline, averaged over five seeds with standard deviation 0.0014.
A deliberately small MLP-Mixer, Mini ALLRAP, still beats all prior methods, implying the loss function rather than model scale carries the geometric prior.
In one-shot settings on single video sequences from ZJU-Mocap and DeformingThings4D, ALLRAP outperforms the baselines without any sequence-specific modules.
The occlusion loss removes the depth mirror-flip ambiguity, so the error can be reported once rather than as the minimum over z and -z.
Because subset selection uses K-nearest neighbors in the predicted 3D space, no skeleton or semantic part annotation is needed; the method applies to arbitrary tracked keypoints.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same local low-rank loss could be applied to dense mesh vertices or point clouds rather than sparse keypoints, potentially lifting continuous surfaces instead of discrete joints; this is an extension beyond the paper's experiments.
Since the prior is entirely in the loss, the MLP-Mixer could be replaced by other high-capacity architectures, and the paper's reported gains would then be attributable to the loss, not to the mixer; the paper does not test this substitution.
If the local low-rank premise is category-agnostic, a model trained on human poses could be fine-tuned on animal data with few examples; the one-shot animal results suggest this may work, but cross-category transfer is not reported.
The batch requirement B>1 means the method cannot lift a single isolated observation at training time; distilling the batch-trained network into a single-frame student would be a natural way to deploy it, a step the paper leaves implicit.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know about this paper because it makes a clean, surprising empirical claim: a generic MLP-Mixer trained with a loss that encourages low-rank structure only on local keypoint subsets beats global low-rank deep NRSfM methods by over 70% on S-Up3D. I read the math and the protocol details, and I'm fairly convinced. The stress-test note worried that the comparison is unfair because ALLRAP uses a one-DOF depth alignment while baselines used a more permissive mirror-flip search. That worry doesn't hold up: the paper explicitly says baselines computed MPJPE twice (z and -z) and ALLRAP does not. So ALLRAP is held to a stricter protocol; the gap is not inflated by this.

What's genuinely new is the formulation of the prior as a training-time loss: select neighborhoods using the network's own predictions, align them, and minimize the log-Gramian determinant of the residual. The ablation in Table 2 is the key evidence: global low-rank gives 0.0308, while local neighborhoods of size 32 give 0.0191 on the same validation split. The occlusion loss is a nice touch and plausibly explains why they can avoid the flip search. The MLP-Mixer result is also a useful data point: 0.24M parameters beat 6.8M MLPs.

The soft spots are in the one-shot experiments. The perspective-camera extension is described in one sentence ('along the camera rays') and never specified. In DeformingThings4D and ZJU-MoCap, they filter keypoints and downsample to 100, but don't say how baselines were handled. Those results are single runs, no variance bars. This is not fatal, but it is underspecified. Also, code is promised but not available, which makes the S-Up3D result harder to fully trust. And the comparison misses several recent deep NRSfM papers (e.g., the 2024 sequence-based methods they cite but don't benchmark). The weakest conceptual assumption—that selected subsets are indeed locally low-rank—is supported by the ablations, so I don't see it as a flaw.

My verdict: this deserves a serious referee. The central claim is important and likely correct. The paper needs a clearer description of the perspective parameterization, a shared pre-processing protocol for the one-shot comparisons, and ideally a code release, before publication. I would bring it to reading group.

Recommendation for the editor: send to peer review. If revisions address the above, this is a strong CVPR/ICCV-type paper.

Referee Report

3 major / 7 minor

Summary. The paper proposes ALLRAP, an unsupervised 2D-3D lifting method for non-rigid objects. It trains a generic MLP-Mixer network with two unsupervised losses: a subset loss that encourages local neighborhoods of keypoints to be approximately low-rank after rigid alignment, and an occlusion loss that weakly enforces a negative correlation between visibility and predicted depth. On the S-Up3D benchmark the method reports an MPJPE of 0.0163, more than 70% lower than the next best baseline, with a standard deviation of 0.0014 over five seeds. The paper also reports one-shot reconstruction results on single sequences from DeformingThings4D and ZJU-MoCap, with lower errors than C3DPO, PAUL, and DeepNRSfM++, and it includes ablations on network architecture and subset selection.

Significance. If the S-Up3D result holds under a shared evaluation protocol, the contribution is significant: it demonstrates that a generic high-capacity network can outperform specialized low-rank architectures when the geometric prior is encoded in the loss, and the local low-rank prior is a compelling alternative to global low-rank constraints. The method is fully unsupervised, handles occluded keypoints, and the S-Up3D result is averaged over five seeds with a small standard deviation. The ablation tables are informative and the framing of the method as matrix inpainting is clear. However, the headline improvement rests on an alignment protocol that is not shown to be identical across methods, and the one-shot comparisons are under-specified; these issues must be resolved before the central claims are established.

major comments (3)

[Section 5, Dataset Metrics; Table 3] The claim of a greater-than-70% reduction on S-Up3D is not yet supported because the text states that MPJPE is computed 'up to one degree of freedom' and that, unlike [20,28,29], the authors do not test both z and -z, but it does not state whether the baseline numbers in Table 3 were recomputed under this same protocol or quoted from the original papers with their own alignment conventions. If the baselines were aligned with full Procrustes, a z-sign search, or no depth-offset removal, the reported gap could be substantially inflated. Please recompute all baselines under the identical one-DOF z-offset alignment, on the same test keypoints, with the same handling of occluded points, and report the alignment details for every method.
[Section 5, DeformingThings4D and ZJU-MoCap; Table 4] The one-shot perspective experiments are underspecified. Section 3 says the perspective case is handled 'along the camera rays', but no equations define how the network input, the predicted coordinates, the subset-loss scaling, or the occlusion loss are expressed in ray coordinates. It is also not stated whether the baselines C3DPO, PAUL, and DeepNRSfM++ were given exactly the same preprocessed inputs (keypoints with visibility above 30%, farthest-point downsampled to 100 keypoints) and the same scale-alignment protocol (centering plus regression scale). Table 4 reports a single run per sequence with no variance; please provide multiple seeds, standard deviations, and a detailed description of the identical protocol applied to all methods.
[Section 3.3.1, Subset selection] The nearest-neighbor subset selection bootstraps the grouping of keypoints from the network's own 3D predictions, so the low-rank loss is applied to neighborhoods that the model itself chooses. The paper validates this empirically in Table 2, but it does not check whether the selected neighborhoods correspond to semantically coherent parts or to artifactually tight clusters created by the loss. Please provide an analysis such as the overlap of selected neighborhoods with ground-truth body parts on S-Up3D, or a comparison with fixed part-based neighborhoods, and discuss the behavior when the network's initial predictions are poor.

minor comments (7)

[Section 3.1] The inpainting rule 'we will only ever use the parts of the outputs that are not already available in the input' is not explicit; please state precisely that visible (x,y) coordinates are taken from the input while occluded (x,y) coordinates and all z coordinates are taken from the network output.
[Equation (8)] The symbol n is used without a formal definition; please specify that the sum runs over the nonzero singular values of the reshaped residual matrix.
[Equation (9)] The cosine similarity is undefined when either v_c or z_c has zero norm; please add an epsilon or a fallback for degenerate batches.
[Table 2] The notation in the first two columns such as '8 ·' and '· 8' is not self-explanatory; please define 'Random' and 'NN' in the caption and clarify the entry format.
[Figure 4] The caption contains a typo: 'Errors are show in using' should read 'Errors are shown using'.
[References] Reference [3] is listed as 'Anon. ... Under submission'; please provide the actual citation if the work is published or identifiable.
[Conclusion] The conclusion states that the implementation will be open sourced; please include a link or supplement in the revision to support reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: unsupervised losses are defined on network outputs and visibility masks, and the only self-referential component (nearest-neighbor subset bootstrapping) is ablated by random-subset experiments.

full rationale

The paper's derivation chain is self-contained: the subset loss (Section 3.3.1, Equations 1-8) and the occlusion loss (Section 3.3.2, Equation 9) are computed entirely from the network's predicted 3D coordinates, the observed 2D keypoints, and the visibility masks, with no ground-truth 3D shape or fitted target used to define the training signal. The nearest-neighbor subset selection is indeed bootstrapped from the model's own predictions, which is a mild self-referential loop, but the random-subset ablations in Table 2 show that random subsets of size 32 still achieve MPJPE 0.0260 versus 0.0191 for learned neighborhoods, so the bootstrap is not load-bearing. The paper cites C3DPO [20], which shares a co-author, but only as a baseline and related work, not as support for ALLRAP's core assumptions. The reported S-Up3D comparison uses a one-degree-of-freedom MPJPE protocol and quotes baseline numbers from prior papers; this raises a legitimate evaluation-protocol comparability concern, but it is a correctness/measurement issue, not circularity, because differing alignment protocols do not make ALLRAP's output a function of its input. No load-bearing step reduces by construction to its own inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The method rests on domain assumptions about low-rank local deformation, visibility-depth correlation, and projection geometry, plus standard linear algebra tools. The free parameters are validation-selected hyperparameters, not fitted target values.

free parameters (4)

subset size k = 32
Chosen on a 50:50 validation split of S-Up3D training data; Table 2 shows neighborhoods of size 32 give the best validation MPJPE.
number of subsets per batch = 10
Empirically chosen; 10 subsets per training batch are averaged into the subset loss.
occlusion loss clamp threshold = -0.05
Hand-chosen threshold to stop biasing shape once the negative visibility-depth correlation is achieved.
MLP-Mixer depth and token width = 32 and 32
Validation sweep over depth 8 to 32 and token width 32; depth 32 was selected for the final experiments.

assumptions (5)

domain assumption Keypoints are semantically corresponding across observations and form a sparse or semi-dense shape representation.
The problem statement and datasets define 2D keypoints with known correspondence; all compared methods rely on this.
domain assumption Local neighborhoods of keypoints have approximately low-rank non-rigid motion after rigid alignment.
Central premise of the subset loss in Section 3.3.1; not proven, only validated empirically.
domain assumption Visibility is weakly negatively correlated with depth for keypoints under self-occlusion.
Basis of the occlusion loss in Section 3.3.2; the authors note exceptions when whole body parts are occluded.
domain assumption Orthographic projection, or normalized device coordinate normalization, makes camera rays parallel so depth is the only missing coordinate.
Used to justify predicting x, y, z directly; the perspective treatment is only sketched in Section 3.
standard math Standard matrix factorization and Kabsch-Umeyama alignment behave as described.
Used in Equations (1) through (7); standard tools, no formal proof provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Unsupervised 2D-3D lifting of non-rigid objects using local constraints." pith.science (2026). https://pith.science/paper/KE5FESHP

@misc{pith2026250419227,
  author       = {Pith},
  title        = {Pith review of: Unsupervised 2D-3D lifting of non-rigid objects using local constraints},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KE5FESHP}},
  note         = {Machine review of arXiv:2504.19227}
}

read the original abstract

For non-rigid objects, predicting the 3D shape from 2D keypoint observations is ill-posed due to occlusions, and the need to disentangle changes in viewpoint and changes in shape. This challenge has often been addressed by embedding low-rank constraints into specialized models. These models can be hard to train, as they depend on finding a canonical way of aligning observations, before they can learn detailed geometry. These constraints have limited the reconstruction quality. We show that generic, high capacity models, trained with an unsupervised loss, allow for more accurate predicted shapes. In particular, applying low-rank constraints to localized subsets of the full shape allows the high capacity to be suitably constrained. We reduce the state-of-the-art reconstruction error on the S-Up3D dataset by over 70%.

Figures

Figures reproduced from arXiv: 2504.19227 by the authors.

**Figure 1.** 3D Reconstruction for partially-occluded 2D semantic keypoints. With an orthographic camera model, for visible points (v = 1) we know the screen coordinates (x, y) and must predict only the depth z. For occluded points (v = 0), we must predict all three (x, y, z)-coordinates. At test time, a trained deep network predicts the unknown coordinates. The network is generic, without any kind of low-rank constraints built-… view at source ↗

**Figure 2.** Unsupervised losses Starting from a randomly-initialized generic deep network, we iteratively learn from batches of partiallyoccluded 2D-annotated training samples. Our training use two unsupervised, batch-wise losses. The subset loss, see Section 3.3.1, acts on the batch of noisy 3D reconstructions. It selects a subset of nearby keypoints, aligns the sub-shapes by rotation and translation, and finally measures the… view at source ↗

**Figure 3.** Subset loss training-time efficiency Plotting the results from [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: ALLRAP S-Up3D test-set reconstructions. (a) We break down the MPJPE error into errors in the camera plane due to occlusion, and errors in the predicted depths. (b) shows a test case with median errors in both components. (c) show the test cases with maximum errors in t…

**Figure 5.** Figure 5: ALLRAP DeformingThings4D one-shot reconstruction Results from training ALLRAP on a sequence of 34 frames from the DeformingThings4D dataset. The reconstructions are shown from a top-down view, with errors shown with red lines. Sequence #Frames C3DPO[20] PAUL[28] Deep N…

**Figure 6.** Figure 6: ALLRAP ZJU-MoCap one-shot reconstructions Results from training ALLRAP on a single sequence of 570 frames from the ZJU-MoCap dataset annotated using [3]. Errors compared to multicamera reconstruction are shown using red lines. picked 100 keypoints per sequence using fa…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 29 canonical work pages

[1]

Nonrigid structure from motion in trajectory space

Ijaz Akhter, Yaser Sheikh, Sohaib Khan, and Takeo Kanade. Nonrigid structure from motion in trajectory space. In Ad- vances in Neural Information Processing Systems . Curran Associates, Inc., 2008. 2

work page 2008
[2]

Trajectory space: A dual representation for nonrigid struc- ture from motion

Ijaz Akhter, Yaser Sheikh, Sohaib Khan, and Takeo Kanade. Trajectory space: A dual representation for nonrigid struc- ture from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(7):1442–1456, 2011. 2

work page 2011
[3]

Real-time volumetric rendering of dynamic humans

Anon. Real-time volumetric rendering of dynamic humans. Under submission, 2023. 6, 8

work page 2023
[4]

Recovering non-rigid 3d shape from image streams

Christoph Bregler, Aaron Hertzmann, and Henning Bier- mann. Recovering non-rigid 3d shape from image streams. In 2000 Conference on Computer Vision and Pattern Recog- nition (CVPR 2000), 13-15 June 2000, Hilton Head, SC, USA, pages 2690–2696. IEEE Computer Society, 2000. 1, 2

work page 2000
[5]

End-to-end learning of multi-category 3d pose and shape estimation

Yigit Baran Can, Alexander Liniger, Danda Pani Paudel, and Luc Van Gool. End-to-end learning of multi-category 3d pose and shape estimation. In 33rd British Machine Vision Conference 2022, BMVC 2022, London, UK, November 21- 24, 2022, page 200. BMV A Press, 2022. 2, 6

work page 2022
[6]

Reconstruct as far as you can: Consensus of non-rigid re- construction from feasible regions

Geonho Cha, Minsik Lee, Junchan Cho, and Songhwai Oh. Reconstruct as far as you can: Consensus of non-rigid re- construction from feasible regions. IEEE Transactions on Pattern Analysis and Machine Intelligence, PP:1–1, 2019. 2

work page 2019
[7]

A simple prior- free method for non-rigid structure-from-motion factoriza- tion

Yuchao Dai, Hongdong Li, and Mingyi He. A simple prior- free method for non-rigid structure-from-motion factoriza- tion. In 2012 IEEE Conference on Computer Vision and Pat- tern Recognition, pages 2018–2025, 2012. 2

work page 2012
[8]

Deep Non-rigid Structure-from-Motion Revisited: Canonicalization and Sequence Modeling

Hui Deng, Jiawei Shi, Zhen Qin, Yiran Zhong, and Yuchao Dai. Deep non-rigid structure-from-motion re- visited: Canonicalization and sequence modeling. arXiv preprint arXiv:2412.07230, 2024. 2, 3

work page Pith review arXiv 2024

Show all 31 references

[9]

Deep non-rigid structure-from- motion: A sequence-to-sequence translation perspective

Hui Deng, Tong Zhang, Yuchao Dai, Jiawei Shi, Yiran Zhong, and Hongdong Li. Deep non-rigid structure-from- motion: A sequence-to-sequence translation perspective. IEEE Transactions on Pattern Analysis and Machine Intel- ligence, 46(12):10814–10828, 2024. 2, 3

2024
[10]

Piece- wise quadratic reconstruction of non-rigid surfaces from monocular sequences

Jo ˜ao Fayad, Lourdes Agapito, and Alessio Del Bue. Piece- wise quadratic reconstruction of non-rigid surfaces from monocular sequences. pages 297–310, 2010. 2

2010
[11]

Grouping-based low-rank trajectory comple- tion and 3d reconstruction

Katerina Fragkiadaki, Marta Salas, Pablo Arbelaez, and Ji- tendra Malik. Grouping-based low-rank trajectory comple- tion and 3d reconstruction. In Advances in Neural Informa- tion Processing Systems. Curran Associates, Inc., 2014. 6

2014
[12]

A solution for the best rotation to re- late two sets of vectors

Wolfgang Kabsch. A solution for the best rotation to re- late two sets of vectors. Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crys- tallography, 32(5):922–923, 1976. 4

1976
[13]

W. Kabsch. A discussion of the solution for the best rotation to relate two sets of vectors. Acta Crystallographica Section A, 34(5):827–828, 1978. 4

1978
[14]

Deep non-rigid structure from motion

Chen Kong and Simon Lucey. Deep non-rigid structure from motion. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), 2019. 2

2019
[15]

Non-rigid structure from motion: Prior- free factorization method revisited

Suryansh Kumar. Non-rigid structure from motion: Prior- free factorization method revisited. InThe IEEE Winter Con- ference on Applications of Computer Vision (WACV), pages 51–60, 2020. 2

2020
[16]

Black, and Peter V

Christoph Lassner, Javier Romero, Martin Kiefel, Federica Bogo, Michael J. Black, and Peter V . Gehler. Unite the peo- ple: Closing the loop between 3d and 2d human representa- tions. In IEEE Conf. on Computer Vision and Pattern Recog- nition (CVPR), 2017. 6

2017
[17]

Consensus of non-rigid reconstructions

Minsik Lee, Jungchan Cho, and Songhwai Oh. Consensus of non-rigid reconstructions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 2

2016
[18]

4dcomplete: Non-rigid motion esti- mation beyond the observable surface

Yang Li, Hikari Takehara, Takafumi Taketomi, Bo Zheng, , and Matthias Nießner. 4dcomplete: Non-rigid motion esti- mation beyond the observable surface. IEEE International Conference on Computer Vision (ICCV), 2021. 6

2021
[19]

Do you even need attention? a stack of feed-forward layers does surprisingly well on imagenet

Luke Melas-Kyriazi. Do you even need attention? a stack of feed-forward layers does surprisingly well on imagenet. arxiv, 2021. 2, 3, 5

2021
[20]

C3DPO: Canonical 3d pose networks for non-rigid structure from motion

David Novotny, Nikhila Ravi, Benjamin Graham, Natalia Neverova, and Andrea Vedaldi. C3DPO: Canonical 3d pose networks for non-rigid structure from motion. In Proceed- ings of the IEEE International Conference on Computer Vi- sion, 2019. 2, 3, 5, 6, 7, 8

2019
[21]

Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans

Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In CVPR,
[22]

Re-thinking non-rigid structure from motion

Vincent Rabaud and Serge Belongie. Re-thinking non-rigid structure from motion. In 2008 IEEE Conference on Com- puter Vision and Pattern Recognition, pages 1–8, 2008. 2

2008
[23]

Ilya O. Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lu- cas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. Mlp-mixer: An all-mlp ar- chitecture for vision. CoRR, abs/2105.01601, ...

2021 arXiv
[24]

Shape and motion from image streams under orthography: a factorization method

Carlo Tomasi and Takeo Kanade. Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis., 9(2):137–154, 1992. 2, 4

1992
[25]

Nonrigid structure-from-motion: Estimating shape and mo- tion with hierarchical priors

Lorenzo Torresani, Aaron Hertzmann, and Chris Bregler. Nonrigid structure-from-motion: Estimating shape and mo- tion with hierarchical priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(5):878–892, 2008. 6

2008
[26]

S. Umeyama. Least-squares estimation of transformation pa- rameters between two point patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1991. 4

1991
[27]

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998–6008, 2017. 4

2017
[28]

Paul: Procrustean au- toencoder for unsupervised lifting

Chaoyang Wang and Simon Lucey. Paul: Procrustean au- toencoder for unsupervised lifting. 2021 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) , pages 434–443, 2021. 2, 3, 6, 7, 8

2021
[29]

Deep nrsfm++: Towards unsupervised 2d-3d lifting in the wild

Chaoyang Wang, Chen-Hsuan Lin, and Simon Lucey. Deep nrsfm++: Towards unsupervised 2d-3d lifting in the wild. In 8th International Conference on 3D Vision, 3DV 2020, Virtual Event, Japan, November 25-28, 2020 , pages 12–22. IEEE, 2020. 2, 6, 7, 8

2020
[30]

Temporal-aware neural network for dense non-rigid structure from motion

Yaming Wang, Dawei Xu, Wenqing Huang, Xiaoping Ye, and Mingfeng Jiang. Temporal-aware neural network for dense non-rigid structure from motion. Electronics, 12(18): 3942, 2023

2023
[31]

Mhr-net: Multiple-hypothesis reconstruction of non-rigid shapes from 2d views

Haitian Zeng, Xin Yu, Jiaxu Miao, and Yi Yang. Mhr-net: Multiple-hypothesis reconstruction of non-rigid shapes from 2d views. In European Conference on Computer Vision , pages 1–17. Springer, 2022. 2

2022

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Nonrigid structure from motion in trajectory space

Ijaz Akhter, Yaser Sheikh, Sohaib Khan, and Takeo Kanade. Nonrigid structure from motion in trajectory space. In Ad- vances in Neural Information Processing Systems . Curran Associates, Inc., 2008. 2

work page 2008

[2] [2]

Trajectory space: A dual representation for nonrigid struc- ture from motion

Ijaz Akhter, Yaser Sheikh, Sohaib Khan, and Takeo Kanade. Trajectory space: A dual representation for nonrigid struc- ture from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(7):1442–1456, 2011. 2

work page 2011

[3] [3]

Real-time volumetric rendering of dynamic humans

Anon. Real-time volumetric rendering of dynamic humans. Under submission, 2023. 6, 8

work page 2023

[4] [4]

Recovering non-rigid 3d shape from image streams

Christoph Bregler, Aaron Hertzmann, and Henning Bier- mann. Recovering non-rigid 3d shape from image streams. In 2000 Conference on Computer Vision and Pattern Recog- nition (CVPR 2000), 13-15 June 2000, Hilton Head, SC, USA, pages 2690–2696. IEEE Computer Society, 2000. 1, 2

work page 2000

[5] [5]

End-to-end learning of multi-category 3d pose and shape estimation

Yigit Baran Can, Alexander Liniger, Danda Pani Paudel, and Luc Van Gool. End-to-end learning of multi-category 3d pose and shape estimation. In 33rd British Machine Vision Conference 2022, BMVC 2022, London, UK, November 21- 24, 2022, page 200. BMV A Press, 2022. 2, 6

work page 2022

[6] [6]

Reconstruct as far as you can: Consensus of non-rigid re- construction from feasible regions

Geonho Cha, Minsik Lee, Junchan Cho, and Songhwai Oh. Reconstruct as far as you can: Consensus of non-rigid re- construction from feasible regions. IEEE Transactions on Pattern Analysis and Machine Intelligence, PP:1–1, 2019. 2

work page 2019

[7] [7]

A simple prior- free method for non-rigid structure-from-motion factoriza- tion

Yuchao Dai, Hongdong Li, and Mingyi He. A simple prior- free method for non-rigid structure-from-motion factoriza- tion. In 2012 IEEE Conference on Computer Vision and Pat- tern Recognition, pages 2018–2025, 2012. 2

work page 2012

[8] [8]

Deep Non-rigid Structure-from-Motion Revisited: Canonicalization and Sequence Modeling

Hui Deng, Jiawei Shi, Zhen Qin, Yiran Zhong, and Yuchao Dai. Deep non-rigid structure-from-motion re- visited: Canonicalization and sequence modeling. arXiv preprint arXiv:2412.07230, 2024. 2, 3

work page Pith review arXiv 2024